[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:38 dist-test-slave-tfpf krb5kdc[17940](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:38 dist-test-slave-tfpf krb5kdc[17940](info): set up 2 sockets
Aug 12 01:08:38 dist-test-slave-tfpf krb5kdc[17940](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 01:08:40.420866 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.543s user 0.002s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:08:40 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:08:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:08:40.586287 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34635
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:34635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:40.697244 17967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:40.697530 17967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:40.697572 17967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:40.698863 17967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:08:40.698942 17967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:40.698967 17967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:40.698992 17967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:08:40.699020 17967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:08:40.701045 17967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:34635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:40.701560 17967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:40.701896 17967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:40.710784 17980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:40.711005 17977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:40.711969 17967 server_base.cc:1034] running on GCE node
W20260812 01:08:40.712348 17975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:40.712795 17967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:40.713299 17967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:40.714509 17967 hybrid_clock.cc:648] HybridClock initialized: now 1786496920714447 us; error 159 us; skew 500 ppm
Aug 12 01:08:40 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:40.722203 17967 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:08:40.722884 17967 webserver.cc:460] Webserver started at http://127.17.130.62:32841/ using document root <none> and password file <none>
I20260812 01:08:40.723304 17967 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:40.723392 17967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:40.723691 17967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:40.725126 17967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "e2562bf344f64c75b2b95c652af90b1e"
format_stamp: "Formatted at 2026-08-12 01:08:40 on dist-test-slave-tfpf"
server_key: "d1d8d342067acdf35f9390e4fa6d928f"
server_key_iv: "b411f1ccd3143a9f92de7d8800471516"
server_key_version: "encryptionkey@0"
I20260812 01:08:40.725621 17967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "e2562bf344f64c75b2b95c652af90b1e"
format_stamp: "Formatted at 2026-08-12 01:08:40 on dist-test-slave-tfpf"
server_key: "d1d8d342067acdf35f9390e4fa6d928f"
server_key_iv: "b411f1ccd3143a9f92de7d8800471516"
server_key_version: "encryptionkey@0"
I20260812 01:08:40.727449 17967 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 01:08:40.728542 17992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:40.728922 17967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:40.729204 17967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "e2562bf344f64c75b2b95c652af90b1e"
format_stamp: "Formatted at 2026-08-12 01:08:40 on dist-test-slave-tfpf"
server_key: "d1d8d342067acdf35f9390e4fa6d928f"
server_key_iv: "b411f1ccd3143a9f92de7d8800471516"
server_key_version: "encryptionkey@0"
I20260812 01:08:40.729310 17967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:40.756649 17967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:40.759851 17967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:40.760156 17967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:40.767838 17967 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:34635
I20260812 01:08:40.767944 18089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:34635 every 8 connection(s)
I20260812 01:08:40.768463 17967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:08:40.770357 18090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:40.773624 18090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e: Bootstrap starting.
I20260812 01:08:40.774772 18090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:40.775125 18090 log.cc:824] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:40.776265 18090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e: No bootstrap required, opened a new log
I20260812 01:08:40.776782 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 17967
I20260812 01:08:40.776904 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:08:40.777788 17928 external_mini_cluster.cc:1428] Setting key fbf2f9682c50e7d975b9baced047b8a5
I20260812 01:08:40.778985 18090 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2562bf344f64c75b2b95c652af90b1e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34635 } }
I20260812 01:08:40.779178 18090 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:40.779206 18090 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2562bf344f64c75b2b95c652af90b1e, State: Initialized, Role: FOLLOWER
I20260812 01:08:40.779374 18090 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2562bf344f64c75b2b95c652af90b1e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34635 } }
I20260812 01:08:40.779471 18090 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:08:40.779510 18090 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:08:40.779561 18090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:40.781328 18090 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2562bf344f64c75b2b95c652af90b1e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34635 } }
I20260812 01:08:40.781476 18090 leader_election.cc:302] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2562bf344f64c75b2b95c652af90b1e; no voters:
I20260812 01:08:40.781720 18090 leader_election.cc:288] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:08:40.782040 18095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:40.782064 18090 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:08:40.782461 18095 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [term 1 LEADER]: Becoming Leader. State: Replica: e2562bf344f64c75b2b95c652af90b1e, State: Running, Role: LEADER
I20260812 01:08:40.782608 18095 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2562bf344f64c75b2b95c652af90b1e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34635 } }
I20260812 01:08:40.783350 18098 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2562bf344f64c75b2b95c652af90b1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2562bf344f64c75b2b95c652af90b1e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34635 } } }
I20260812 01:08:40.783468 18098 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:40.783728 18100 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2562bf344f64c75b2b95c652af90b1e. Latest consensus state: current_term: 1 leader_uuid: "e2562bf344f64c75b2b95c652af90b1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2562bf344f64c75b2b95c652af90b1e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34635 } } }
I20260812 01:08:40.783819 18100 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e [sys.catalog]: This master's current role is: LEADER
W20260812 01:08:40.785264 18119 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:08:40.785336 18119 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:08:40.785399 18114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:08:40.785722 18114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:08:40.787813 18114 catalog_manager.cc:1347] Generated new cluster ID: ba9af9d4cc6d4b72a4534ac9d1e3db71
I20260812 01:08:40.787881 18114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:08:40 dist-test-slave-tfpf krb5kdc[17940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:40.801347 18114 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:08:40.801400 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:40.779154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35104 (local address 127.17.130.62:34635)
0812 01:08:40.779418 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:40.779429 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:40.779921 (+ 492us) server_negotiation.cc:407] Connection header received
0812 01:08:40.782907 (+ 2986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:40.782918 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:40.783019 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:40.783172 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:40.785994 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:40.787250 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:40.788793 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:40.789561 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:40.789823 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:40.795015 (+ 5192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:40.795027 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:40.795027 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:40.795054 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:40.797930 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:40.798768 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:40.798771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:40.798772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:40.799124 (+ 352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:40.799540 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:40.799540 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:40.799541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:40.799887 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:40.800454 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:40.800803 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 01:08:40.800971 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":150,"threads_started":1}
I20260812 01:08:40.802142 18114 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:08:40.811324 18114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2562bf344f64c75b2b95c652af90b1e: Generated new TSK 0
I20260812 01:08:40.811584 18114 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:08:40.853331 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34635
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:40.954351 18135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:40.954592 18135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:40.954620 18135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:40.955783 18135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:40.955827 18135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:40.955854 18135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:08:40.957520 18135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:40.957765 18135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:40.957962 18135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:40.963856 18146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:40.963908 18143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:40.963841 18142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:40.964186 18135 server_base.cc:1034] running on GCE node
I20260812 01:08:40.964423 18135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:40.964717 18135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:40.965924 18135 hybrid_clock.cc:648] HybridClock initialized: now 1786496920965866 us; error 81 us; skew 500 ppm
Aug 12 01:08:40 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:40.969529 18135 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:08:40.970350 18135 webserver.cc:460] Webserver started at http://127.17.130.1:36753/ using document root <none> and password file <none>
I20260812 01:08:40.970721 18135 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:40.970808 18135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:40.970961 18135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:40.972766 18135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "8f9cfa2aa567477b8ccf8b6cf6e8eb1c"
format_stamp: "Formatted at 2026-08-12 01:08:40 on dist-test-slave-tfpf"
server_key: "71aab87d5a5552ec0cc6fa78960c9f39"
server_key_iv: "9ba25ba901d455cae1272c5ecfa3b98b"
server_key_version: "encryptionkey@0"
I20260812 01:08:40.973376 18135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "8f9cfa2aa567477b8ccf8b6cf6e8eb1c"
format_stamp: "Formatted at 2026-08-12 01:08:40 on dist-test-slave-tfpf"
server_key: "71aab87d5a5552ec0cc6fa78960c9f39"
server_key_iv: "9ba25ba901d455cae1272c5ecfa3b98b"
server_key_version: "encryptionkey@0"
I20260812 01:08:40.975170 18135 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:08:40.976261 18158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:40.976581 18135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:40.976739 18135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "8f9cfa2aa567477b8ccf8b6cf6e8eb1c"
format_stamp: "Formatted at 2026-08-12 01:08:40 on dist-test-slave-tfpf"
server_key: "71aab87d5a5552ec0cc6fa78960c9f39"
server_key_iv: "9ba25ba901d455cae1272c5ecfa3b98b"
server_key_version: "encryptionkey@0"
I20260812 01:08:40.976832 18135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:40.989362 18135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:40.992113 18135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:40.992311 18135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:40.992676 18135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:40.993105 18135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:40.993173 18135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:40.993216 18135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:40.993245 18135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.006390 18135 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:35373
I20260812 01:08:41.006440 18346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:35373 every 8 connection(s)
I20260812 01:08:41.006940 18135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:08:41.011478 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18135
I20260812 01:08:41.011579 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:08:41.011791 17928 external_mini_cluster.cc:1428] Setting key 5b809257707f78c626ecd052bc26b513
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496920, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:41.018708 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.008072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:43039 (local address 127.17.130.62:34635)
0812 01:08:41.008166 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:41.008169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:41.008591 (+ 422us) server_negotiation.cc:407] Connection header received
0812 01:08:41.009143 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:41.009145 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:41.009168 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:41.009233 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:41.010580 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.011250 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.012495 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.012720 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.012800 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.015384 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:41.015386 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:41.015386 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:41.015396 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:41.017022 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.017571 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.017573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.017573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.017646 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.017946 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.017947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.017947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.018081 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:41.018167 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:41.018484 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 01:08:41.018607 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:08:41.018884 18349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.008040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34635 (local address 127.17.130.1:43039)
0812 01:08:41.008440 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:41.008491 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:41.009006 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:41.009301 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:41.009305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:41.010196 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:41.010434 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:41.010440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.011381 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.011383 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:41.012403 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:41.012406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.012812 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.012813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:41.012966 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.013639 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:41.013649 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:41.015272 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:41.017197 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:41.017208 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:41.017211 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:41.017485 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:41.017720 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:41.017721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:41.017722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:41.017856 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:41.018158 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:41.018166 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:41.018329 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 01:08:41.018445 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 01:08:41.018627 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":185,"threads_started":1}
I20260812 01:08:41.019425 18347 heartbeater.cc:347] Connected to a master server at 127.17.130.62:34635
I20260812 01:08:41.019577 18347 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:41.019837 18347 heartbeater.cc:511] Master 127.17.130.62:34635 requested a full tablet report, sending...
I20260812 01:08:41.020764 18022 ts_manager.cc:194] Registered new tserver with Master: 8f9cfa2aa567477b8ccf8b6cf6e8eb1c (127.17.130.1:35373)
I20260812 01:08:41.021626 18022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:43039
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:08:41.046639 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34635
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:41.144124 18352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:41.144384 18352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:41.144414 18352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:41.145684 18352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:41.145735 18352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:41.145766 18352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:08:41.147346 18352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:41.147574 18352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:41.147782 18352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:41.154901 18364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:41.154974 18361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:41.155186 18359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:41.155401 18352 server_base.cc:1034] running on GCE node
I20260812 01:08:41.155735 18352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:41.155977 18352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:41.157167 18352 hybrid_clock.cc:648] HybridClock initialized: now 1786496921157137 us; error 48 us; skew 500 ppm
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:41.160495 18352 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:08:41.161123 18352 webserver.cc:460] Webserver started at http://127.17.130.2:46437/ using document root <none> and password file <none>
I20260812 01:08:41.161393 18352 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:41.161445 18352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:41.161607 18352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:41.162911 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "03b05fccbd4d45a783a41415381a4a90"
format_stamp: "Formatted at 2026-08-12 01:08:41 on dist-test-slave-tfpf"
server_key: "9cf1b56074ea98693e7e36bdea6c3685"
server_key_iv: "1ae1ae7219d6d2cf7c70b64df9361ef8"
server_key_version: "encryptionkey@0"
I20260812 01:08:41.163467 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "03b05fccbd4d45a783a41415381a4a90"
format_stamp: "Formatted at 2026-08-12 01:08:41 on dist-test-slave-tfpf"
server_key: "9cf1b56074ea98693e7e36bdea6c3685"
server_key_iv: "1ae1ae7219d6d2cf7c70b64df9361ef8"
server_key_version: "encryptionkey@0"
I20260812 01:08:41.165475 18352 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:08:41.166780 18374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.167176 18352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:41.167282 18352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "03b05fccbd4d45a783a41415381a4a90"
format_stamp: "Formatted at 2026-08-12 01:08:41 on dist-test-slave-tfpf"
server_key: "9cf1b56074ea98693e7e36bdea6c3685"
server_key_iv: "1ae1ae7219d6d2cf7c70b64df9361ef8"
server_key_version: "encryptionkey@0"
I20260812 01:08:41.167392 18352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:41.181618 18352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:41.184722 18352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:41.184933 18352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:41.185385 18352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:41.185813 18352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:41.185875 18352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.185940 18352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:41.185979 18352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.199463 18352 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:45019
I20260812 01:08:41.199512 18562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:45019 every 8 connection(s)
I20260812 01:08:41.199954 18352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:08:41.205376 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18352
I20260812 01:08:41.205503 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:08:41.205716 17928 external_mini_cluster.cc:1428] Setting key b6db9f4a5ec0b24314541c97c0461caf
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:41.213521 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.200908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34635 (local address 127.17.130.2:59079)
0812 01:08:41.201230 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:41.201284 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:41.201821 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:41.202456 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:41.202460 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:41.203244 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:41.203590 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:41.203596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.204397 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.204398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:41.205406 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:41.205410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.205935 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.205936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:41.206114 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.206746 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:41.206756 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:41.208750 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:41.211140 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:41.211153 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:41.211156 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:41.211538 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:41.211832 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:41.211833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:41.211833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:41.212007 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:41.212522 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:41.212531 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:41.212733 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 01:08:41.212894 (+ 161us) client_negotiation.cc:240] Negotiation successful
0812 01:08:41.213180 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":108,"threads_started":1}
I20260812 01:08:41.213551 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.201029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:59079 (local address 127.17.130.62:34635)
0812 01:08:41.201163 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:41.201166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:41.201397 (+ 231us) server_negotiation.cc:407] Connection header received
0812 01:08:41.202218 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:41.202219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:41.202239 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:41.202316 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:41.203728 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.204280 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.205532 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.205820 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.205920 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.208898 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:41.208900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:41.208900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:41.208919 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:41.211025 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.211670 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.211671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.211672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.211744 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.212268 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.212269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.212270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.212419 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:41.212494 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:41.213054 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 01:08:41.213443 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 01:08:41.214203 18563 heartbeater.cc:347] Connected to a master server at 127.17.130.62:34635
I20260812 01:08:41.214329 18563 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:41.214589 18563 heartbeater.cc:511] Master 127.17.130.62:34635 requested a full tablet report, sending...
I20260812 01:08:41.215153 18022 ts_manager.cc:194] Registered new tserver with Master: 03b05fccbd4d45a783a41415381a4a90 (127.17.130.2:45019)
I20260812 01:08:41.215610 18022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:59079
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:08:41.242887 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34635
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:41.339819 18571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:41.340085 18571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:41.340162 18571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:41.341502 18571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:41.341575 18571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:41.341627 18571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:08:41.343438 18571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34635
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:41.343756 18571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:41.344054 18571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:41.350268 18577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:41.350317 18578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:41.350672 18583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:41.351739 18571 server_base.cc:1034] running on GCE node
I20260812 01:08:41.352027 18571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:41.352334 18571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:41.353538 18571 hybrid_clock.cc:648] HybridClock initialized: now 1786496921353519 us; error 34 us; skew 500 ppm
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:41.358139 18571 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:08:41.358713 18571 webserver.cc:460] Webserver started at http://127.17.130.3:41525/ using document root <none> and password file <none>
I20260812 01:08:41.359037 18571 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:41.359122 18571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:41.359301 18571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:41.361395 18571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "e6e181456e974e2db60bdcb9d798b6d6"
format_stamp: "Formatted at 2026-08-12 01:08:41 on dist-test-slave-tfpf"
server_key: "95aa2b13bcd1f9b010578a3801efa214"
server_key_iv: "df40f7160e53253a90b1532851163f30"
server_key_version: "encryptionkey@0"
I20260812 01:08:41.362174 18571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "e6e181456e974e2db60bdcb9d798b6d6"
format_stamp: "Formatted at 2026-08-12 01:08:41 on dist-test-slave-tfpf"
server_key: "95aa2b13bcd1f9b010578a3801efa214"
server_key_iv: "df40f7160e53253a90b1532851163f30"
server_key_version: "encryptionkey@0"
I20260812 01:08:41.364833 18571 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 01:08:41.366056 18593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.366501 18571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:41.366678 18571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "e6e181456e974e2db60bdcb9d798b6d6"
format_stamp: "Formatted at 2026-08-12 01:08:41 on dist-test-slave-tfpf"
server_key: "95aa2b13bcd1f9b010578a3801efa214"
server_key_iv: "df40f7160e53253a90b1532851163f30"
server_key_version: "encryptionkey@0"
I20260812 01:08:41.366793 18571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:41.382045 18571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:41.385017 18571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:41.385286 18571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:41.385586 18571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:41.386023 18571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:41.386085 18571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.386132 18571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:41.386160 18571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:41.397567 18571 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:38921
I20260812 01:08:41.397614 18764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:38921 every 8 connection(s)
I20260812 01:08:41.398007 18571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:08:41.401261 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18571
I20260812 01:08:41.401343 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:08:41.401513 17928 external_mini_cluster.cc:1428] Setting key bf80013996fbd39a3a7da0122bc5883e
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:41.409667 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.398911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:47979 (local address 127.17.130.62:34635)
0812 01:08:41.399089 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:41.399092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:41.399267 (+ 175us) server_negotiation.cc:407] Connection header received
0812 01:08:41.399864 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:41.399865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:41.399879 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:41.399935 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:41.401007 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.401676 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.402892 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.403102 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.403195 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.406206 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:41.406207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:41.406207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:41.406216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:41.407888 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.408415 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.408416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.408417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.408482 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.408858 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.408859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.408859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.408997 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:41.409167 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:41.409502 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 01:08:41.409594 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 01:08:41.409926 18770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.398882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34635 (local address 127.17.130.3:47979)
0812 01:08:41.399125 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:41.399181 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:41.399742 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:41.400021 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:41.400026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:41.400680 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:41.400899 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:41.400903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.401774 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.401775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:41.402788 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:41.402791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.403208 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.403209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:41.403530 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.404150 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:41.404160 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:41.406027 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:41.408020 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:41.408029 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:41.408031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:41.408308 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:41.408583 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:41.408584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:41.408584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:41.408736 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:41.409162 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:41.409177 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:41.409343 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 01:08:41.409470 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 01:08:41.409628 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":118,"threads_started":1}
I20260812 01:08:41.410784 18767 heartbeater.cc:347] Connected to a master server at 127.17.130.62:34635
I20260812 01:08:41.410975 18767 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:41.411257 18767 heartbeater.cc:511] Master 127.17.130.62:34635 requested a full tablet report, sending...
I20260812 01:08:41.411895 18022 ts_manager.cc:194] Registered new tserver with Master: e6e181456e974e2db60bdcb9d798b6d6 (127.17.130.3:38921)
I20260812 01:08:41.412278 18022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:47979
I20260812 01:08:41.414428 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:08:41.445513 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.435174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.17.130.1:35373)
0812 01:08:41.435357 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:41.435364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:41.435791 (+ 427us) server_negotiation.cc:407] Connection header received
0812 01:08:41.435926 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:41.435931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:41.435970 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:41.436093 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:41.436903 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.437957 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.438905 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.439265 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.439354 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.442229 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:41.442241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:41.442241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:41.442256 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:41.444124 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.444578 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.444581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.444582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.444660 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.444867 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.444868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.444869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.445026 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:41.445230 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:41.445279 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 01:08:41.445367 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
W20260812 01:08:41.446065 18202 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57758
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[17940](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:08:41.475771 18775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:41.466616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57770 (local address 127.17.130.1:35373)
0812 01:08:41.466676 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:41.466678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:41.466844 (+ 166us) server_negotiation.cc:407] Connection header received
0812 01:08:41.466938 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:41.466939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:41.466966 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:41.467051 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:41.467779 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.468356 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.469194 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:41.469436 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:41.469507 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:41.472546 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:41.472548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:41.472549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:41.472558 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:41.474332 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.474775 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.474776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.474777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.474863 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:41.475141 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:41.475143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:41.475143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:41.475356 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:41.475467 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:41.475570 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 01:08:41.475652 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:08:41.476815 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18135
I20260812 01:08:41.487030 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18352
I20260812 01:08:41.501829 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18571
I20260812 01:08:41.538792 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 17967
2026-08-12T01:08:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2697 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[18785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[18785](info): set up 2 sockets
Aug 12 01:08:41 dist-test-slave-tfpf krb5kdc[18785](info): commencing operation
krb5kdc: starting...
W20260812 01:08:42.864794 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.007s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:08:42 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:08:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:08:43.000162 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44149
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44149
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:43.089605 18815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:43.089854 18815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:43.089893 18815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:43.091090 18815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:08:43.091168 18815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:43.091195 18815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:43.091224 18815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:08:43.091248 18815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:08:43.092774 18815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:43.093142 18815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:43.093387 18815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:43.098827 18828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:43.098901 18815 server_base.cc:1034] running on GCE node
W20260812 01:08:43.098841 18824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.098908 18826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:43.099457 18815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:43.099840 18815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:43.101012 18815 hybrid_clock.cc:648] HybridClock initialized: now 1786496923100991 us; error 35 us; skew 500 ppm
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:43.103946 18815 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:08:43.104461 18815 webserver.cc:460] Webserver started at http://127.17.130.62:37813/ using document root <none> and password file <none>
I20260812 01:08:43.104671 18815 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:43.104718 18815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:43.104815 18815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:43.106030 18815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "09ae4bd1a5d2444bbacd3550f90bacaa"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "8d76633203cf28c9acf4fb5cfd3d5323"
server_key_iv: "3c9865c0b16a0043e642e7e4d10d6dc3"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.106364 18815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "09ae4bd1a5d2444bbacd3550f90bacaa"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "8d76633203cf28c9acf4fb5cfd3d5323"
server_key_iv: "3c9865c0b16a0043e642e7e4d10d6dc3"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.107781 18815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:08:43.108541 18839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.108825 18815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:08:43.108979 18815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "09ae4bd1a5d2444bbacd3550f90bacaa"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "8d76633203cf28c9acf4fb5cfd3d5323"
server_key_iv: "3c9865c0b16a0043e642e7e4d10d6dc3"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.109071 18815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:43.127246 18815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:43.130020 18815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:43.130239 18815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:43.136276 18815 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44149
I20260812 01:08:43.136308 18926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44149 every 8 connection(s)
I20260812 01:08:43.136811 18815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:08:43.137916 18932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:43.140872 18932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa: Bootstrap starting.
I20260812 01:08:43.141624 18932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:43.141959 18932 log.cc:824] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:43.142982 18932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa: No bootstrap required, opened a new log
I20260812 01:08:43.144812 18932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44149 } }
I20260812 01:08:43.145215 18932 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:43.145272 18932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09ae4bd1a5d2444bbacd3550f90bacaa, State: Initialized, Role: FOLLOWER
I20260812 01:08:43.145462 18932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44149 } }
I20260812 01:08:43.145541 18932 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:08:43.145565 18932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:08:43.145598 18932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:43.146353 18932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44149 } }
I20260812 01:08:43.146461 18932 leader_election.cc:302] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09ae4bd1a5d2444bbacd3550f90bacaa; no voters:
I20260812 01:08:43.146589 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18815
I20260812 01:08:43.146713 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:08:43.146705 18932 leader_election.cc:288] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:08:43.146739 18939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:43.146867 18939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [term 1 LEADER]: Becoming Leader. State: Replica: 09ae4bd1a5d2444bbacd3550f90bacaa, State: Running, Role: LEADER
I20260812 01:08:43.146943 17928 external_mini_cluster.cc:1428] Setting key a75c491829e502e386ded176d7177909
I20260812 01:08:43.146960 18939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44149 } }
I20260812 01:08:43.147032 18932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:08:43.147442 18939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09ae4bd1a5d2444bbacd3550f90bacaa. Latest consensus state: current_term: 1 leader_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44149 } } }
I20260812 01:08:43.147533 18939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:43.147640 18941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09ae4bd1a5d2444bbacd3550f90bacaa" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44149 } } }
I20260812 01:08:43.147722 18941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:43.148290 18950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:08:43.149008 18950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:08:43.151748 18950 catalog_manager.cc:1347] Generated new cluster ID: 3bb53de92b304278bcdb7ab52a6544c9
I20260812 01:08:43.151811 18950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:43.159087 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.147740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.17.130.62:44149)
0812 01:08:43.148046 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.148056 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.148208 (+ 152us) server_negotiation.cc:407] Connection header received
0812 01:08:43.148506 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.148515 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.148604 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.148738 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:43.149883 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.151268 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.152238 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.152722 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.152856 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.155429 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:43.155434 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:43.155434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:43.155449 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:43.157425 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.157861 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.157864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.157865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.157986 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.158211 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.158212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.158212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.158481 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:43.158667 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.158744 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.158902 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260812 01:08:43.162053 18950 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:08:43.162611 18950 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:08:43.172369 18950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09ae4bd1a5d2444bbacd3550f90bacaa: Generated new TSK 0
I20260812 01:08:43.172621 18950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:08:43.213511 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44149
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:43.326902 18986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:43.327185 18986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:43.327242 18986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:43.328501 18986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:43.328577 18986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:43.328627 18986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:08:43.330340 18986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44149
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:43.330636 18986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:43.330893 18986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:43.336599 18994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.336619 18995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.336599 18998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:43.337050 18986 server_base.cc:1034] running on GCE node
I20260812 01:08:43.337467 18986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:43.337879 18986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:43.339088 18986 hybrid_clock.cc:648] HybridClock initialized: now 1786496923339041 us; error 74 us; skew 500 ppm
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:43.342458 18986 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:08:43.343124 18986 webserver.cc:460] Webserver started at http://127.17.130.1:44835/ using document root <none> and password file <none>
I20260812 01:08:43.343357 18986 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:43.343410 18986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:43.343513 18986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:43.344699 18986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "331784e6842746df853c2182331c5d51"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "0fe9ac9d6c9dc08d29b5bffab38890ed"
server_key_iv: "11fa0556b40807a87ff410485fbb7e18"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.345254 18986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "331784e6842746df853c2182331c5d51"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "0fe9ac9d6c9dc08d29b5bffab38890ed"
server_key_iv: "11fa0556b40807a87ff410485fbb7e18"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.347023 18986 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:08:43.348209 19009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.348551 18986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:43.348667 18986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "331784e6842746df853c2182331c5d51"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "0fe9ac9d6c9dc08d29b5bffab38890ed"
server_key_iv: "11fa0556b40807a87ff410485fbb7e18"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.348766 18986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:43.364058 18986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:43.367024 18986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:43.367255 18986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:43.367547 18986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:43.367937 18986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:43.367977 18986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.368002 18986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:43.368016 18986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.375087 18986 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:36697
I20260812 01:08:43.375185 19201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:36697 every 8 connection(s)
I20260812 01:08:43.375550 18986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:08:43.382167 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18986
I20260812 01:08:43.382265 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:08:43.382453 17928 external_mini_cluster.cc:1428] Setting key 25c386b746b7eaa7039f95d099a2bac7
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:43.387586 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.376540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:44313 (local address 127.17.130.62:44149)
0812 01:08:43.376595 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.376597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.376838 (+ 241us) server_negotiation.cc:407] Connection header received
0812 01:08:43.378085 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.378087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.378103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.378179 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:43.379062 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.379700 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.380975 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.381273 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.381342 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.384302 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:43.384304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:43.384304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:43.384313 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:43.386014 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.386575 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.386576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.386576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.386639 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.386931 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.386932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.386932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.387041 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:43.387093 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.387444 (+ 351us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.387517 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:08:43.387933 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.376451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44149 (local address 127.17.130.1:44313)
0812 01:08:43.376706 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:43.376754 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:43.377949 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:43.378288 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:43.378293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:43.378717 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:43.378931 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.378937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.379839 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.379839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.380866 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.380869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.381387 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.381388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.381502 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.382141 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:43.382150 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:43.384199 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:43.386112 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:43.386123 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:43.386126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:43.386487 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:43.386709 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:43.386710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:43.386711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:43.386863 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:43.387108 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:43.387116 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:43.387300 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 01:08:43.387425 (+ 125us) client_negotiation.cc:240] Negotiation successful
0812 01:08:43.387636 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":106,"threads_started":1}
I20260812 01:08:43.388597 19202 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44149
I20260812 01:08:43.388706 19202 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:43.388927 19202 heartbeater.cc:511] Master 127.17.130.62:44149 requested a full tablet report, sending...
I20260812 01:08:43.389693 18863 ts_manager.cc:194] Registered new tserver with Master: 331784e6842746df853c2182331c5d51 (127.17.130.1:36697)
I20260812 01:08:43.390722 18863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:44313
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:08:43.412711 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44149
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:43.508927 19208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:43.509234 19208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:43.509268 19208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:43.510407 19208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:43.510447 19208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:43.510473 19208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:08:43.512050 19208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44149
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:43.512272 19208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:43.512518 19208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:43.518347 19216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.518304 19215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.518304 19219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:43.518842 19208 server_base.cc:1034] running on GCE node
I20260812 01:08:43.519158 19208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:43.519434 19208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:43.520704 19208 hybrid_clock.cc:648] HybridClock initialized: now 1786496923520682 us; error 40 us; skew 500 ppm
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:43.523808 19208 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:08:43.524399 19208 webserver.cc:460] Webserver started at http://127.17.130.2:41815/ using document root <none> and password file <none>
I20260812 01:08:43.524646 19208 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:43.524694 19208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:43.524828 19208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:43.526001 19208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "76bcffdd5796439898cc4d6d8d79d181"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "479edee7727f38109eef1b69106b12f1"
server_key_iv: "504e2e008b45605fbcd38e5754dd8440"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.526396 19208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "76bcffdd5796439898cc4d6d8d79d181"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "479edee7727f38109eef1b69106b12f1"
server_key_iv: "504e2e008b45605fbcd38e5754dd8440"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.527918 19208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:08:43.528818 19227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.529134 19208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:43.529266 19208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "76bcffdd5796439898cc4d6d8d79d181"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "479edee7727f38109eef1b69106b12f1"
server_key_iv: "504e2e008b45605fbcd38e5754dd8440"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.529366 19208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:43.540999 19208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:43.543411 19208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:43.543612 19208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:43.543895 19208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:43.544253 19208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:43.544310 19208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.544353 19208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:43.544384 19208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.551522 19208 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:38935
I20260812 01:08:43.551609 19411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:38935 every 8 connection(s)
I20260812 01:08:43.552314 19208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:08:43.559811 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19208
I20260812 01:08:43.559916 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:08:43.560122 17928 external_mini_cluster.cc:1428] Setting key 6db4f4cd5855123ab4c531433a4138db
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:43.564527 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.553989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:44201 (local address 127.17.130.62:44149)
0812 01:08:43.554073 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.554075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.554276 (+ 201us) server_negotiation.cc:407] Connection header received
0812 01:08:43.555241 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.555242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.555263 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.555337 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:43.556047 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.556583 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.557724 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.558013 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.558149 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.560935 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:43.560937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:43.560938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:43.560948 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:43.562515 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.563147 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.563149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.563150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.563283 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.563765 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.563767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.563767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.563922 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:43.564008 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.564323 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.564431 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:08:43.564785 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.553880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44149 (local address 127.17.130.2:44201)
0812 01:08:43.554083 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:43.554131 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:43.555100 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:43.555424 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:43.555427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:43.555733 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:43.555933 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.555937 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.556696 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.556697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.557611 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.557613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.558193 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.558195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.558323 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.559010 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:43.559020 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:43.560811 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:43.562620 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:43.562630 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:43.562633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:43.562896 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:43.563528 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:43.563529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:43.563530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:43.563675 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:43.563999 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:43.564007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:43.564169 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 01:08:43.564285 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 01:08:43.564466 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
I20260812 01:08:43.565361 19413 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44149
I20260812 01:08:43.565501 19413 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:43.565757 19413 heartbeater.cc:511] Master 127.17.130.62:44149 requested a full tablet report, sending...
I20260812 01:08:43.566286 18863 ts_manager.cc:194] Registered new tserver with Master: 76bcffdd5796439898cc4d6d8d79d181 (127.17.130.2:38935)
I20260812 01:08:43.566716 18863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:44201
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:08:43.593287 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44149
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:43.690377 19419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:43.690608 19419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:43.690634 19419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:43.691846 19419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:43.691895 19419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:43.691926 19419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:08:43.693691 19419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44149
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:43.693982 19419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:43.694272 19419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:43.700624 19430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.700898 19424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:43.700574 19427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:43.701213 19419 server_base.cc:1034] running on GCE node
I20260812 01:08:43.701436 19419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:43.701706 19419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:43.702947 19419 hybrid_clock.cc:648] HybridClock initialized: now 1786496923702921 us; error 61 us; skew 500 ppm
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:43.705953 19419 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:08:43.706660 19419 webserver.cc:460] Webserver started at http://127.17.130.3:42581/ using document root <none> and password file <none>
I20260812 01:08:43.707059 19419 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:43.707147 19419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:43.707319 19419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:43.708555 19419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "d9071ef9ec0647319a8846dbbdcafc20"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "cb5ceeb77c8ccb520349f03604799346"
server_key_iv: "bdbd0501b226556518f4fff469fcf6d7"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.708948 19419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "d9071ef9ec0647319a8846dbbdcafc20"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "cb5ceeb77c8ccb520349f03604799346"
server_key_iv: "bdbd0501b226556518f4fff469fcf6d7"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.710525 19419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:43.711570 19445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.712093 19419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:43.712267 19419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "d9071ef9ec0647319a8846dbbdcafc20"
format_stamp: "Formatted at 2026-08-12 01:08:43 on dist-test-slave-tfpf"
server_key: "cb5ceeb77c8ccb520349f03604799346"
server_key_iv: "bdbd0501b226556518f4fff469fcf6d7"
server_key_version: "encryptionkey@0"
I20260812 01:08:43.712355 19419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:43.724013 19419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:43.726683 19419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:43.726888 19419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:43.727164 19419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:43.727524 19419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:43.727583 19419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.727638 19419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:43.727665 19419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:43.737313 19419 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:37845
I20260812 01:08:43.737355 19625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:37845 every 8 connection(s)
I20260812 01:08:43.737748 19419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:08:43.741683 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19419
I20260812 01:08:43.741845 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:08:43.741976 17928 external_mini_cluster.cc:1428] Setting key e176c49d56a6e1782963da1c2e53b96c
Aug 12 01:08:43 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496923, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:43.749606 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.738759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:33411 (local address 127.17.130.62:44149)
0812 01:08:43.739140 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.739143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.739311 (+ 168us) server_negotiation.cc:407] Connection header received
0812 01:08:43.740340 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.740341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.740356 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.740452 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:43.741429 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.742160 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.743411 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.743612 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.743716 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.746263 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:43.746264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:43.746265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:43.746272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:43.747884 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.748459 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.748461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.748461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.748540 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.748859 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.748860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.748860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.748996 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:43.749117 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.749456 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.749535 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 01:08:43.749771 19633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.738659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44149 (local address 127.17.130.3:33411)
0812 01:08:43.739162 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:43.739210 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:43.740210 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:43.740538 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:43.740546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:43.740979 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:43.741314 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.741320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.742281 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.742282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.743304 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.743307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.743742 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.743743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.743891 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.744612 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:43.744621 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:43.746153 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:43.747990 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:43.747998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:43.748001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:43.748356 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:43.748642 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:43.748643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:43.748644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:43.748769 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:43.749137 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:43.749151 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:43.749292 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 01:08:43.749396 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 01:08:43.749545 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":183,"threads_started":1}
I20260812 01:08:43.750736 19631 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44149
I20260812 01:08:43.750888 19631 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:43.751128 19631 heartbeater.cc:511] Master 127.17.130.62:44149 requested a full tablet report, sending...
I20260812 01:08:43.751641 18863 ts_manager.cc:194] Registered new tserver with Master: d9071ef9ec0647319a8846dbbdcafc20 (127.17.130.3:37845)
I20260812 01:08:43.752174 18863 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:33411
I20260812 01:08:43.754021 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:08:43.763124 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.755966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36702 (local address 127.17.130.62:44149)
0812 01:08:43.756044 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.756046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.756493 (+ 447us) server_negotiation.cc:407] Connection header received
0812 01:08:43.756588 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.756589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.756619 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.756709 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:43.757391 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.757887 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.758616 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.758807 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.758885 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.760277 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:43.760279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:43.760280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:43.760287 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:43.761748 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.762223 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.762224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.762224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.762287 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:43.762501 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:43.762502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:43.762502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:43.762674 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:43.762781 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.762930 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.763017 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:08:43.766127 18863 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:08:43.768224 18863 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:43.776051 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.771182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37845 (local address 127.0.0.1:40744)
0812 01:08:43.771401 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:43.771445 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:43.771664 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:43.772068 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:43.772071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:43.772074 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:43.772223 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.772226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.773838 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.773839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.774921 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.774923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.775637 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.775725 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.775739 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 01:08:43.775812 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 01:08:43.775876 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
I20260812 01:08:43.776136 19652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.771184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:38935 (local address 127.0.0.1:60008)
0812 01:08:43.771725 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:43.771736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:43.771945 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:43.772373 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:43.772374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:43.772376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:43.772412 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.772413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.773544 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.773546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.774836 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.774839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.775781 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.775841 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.775844 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:43.775894 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 01:08:43.775923 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":121,"threads_started":1}
I20260812 01:08:43.776449 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.771531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.17.130.3:37845)
0812 01:08:43.771709 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.771713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.771744 (+ 31us) server_negotiation.cc:407] Connection header received
0812 01:08:43.771817 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.771826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.771854 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.771935 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:43.772319 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.773715 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775031 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.775536 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775628 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.775695 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.776317 (+ 622us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.776363 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
I20260812 01:08:43.776707 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.771298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.17.130.2:38935)
0812 01:08:43.771492 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.771497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.771912 (+ 415us) server_negotiation.cc:407] Connection header received
0812 01:08:43.772042 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.772045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.772061 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.772251 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:43.772610 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.773438 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.774938 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.775688 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775778 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.775835 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.775906 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.776215 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":118,"threads_started":1}
I20260812 01:08:43.776772 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.771820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.17.130.1:36697)
0812 01:08:43.772329 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:43.772332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:43.772340 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:08:43.772373 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:43.772374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:43.772385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:43.772523 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:43.772781 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.773973 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775054 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.775714 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775777 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.775835 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:43.776621 (+ 786us) server_negotiation.cc:299] Negotiation successful
0812 01:08:43.776667 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":104,"threads_started":1}
I20260812 01:08:43.776960 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:43.771474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36697 (local address 127.0.0.1:52794)
0812 01:08:43.772013 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:43.772024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:43.772162 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:43.772635 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:43.772636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:43.772638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:43.772692 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.772693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.774088 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.774090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.774954 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:43.774957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:43.775875 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:43.775877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:43.776371 (+ 494us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:43.776381 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 01:08:43.776438 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:08:43.776489 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":174,"threads_started":1}
I20260812 01:08:43.777364 19311 tablet_service.cc:1467] Processing CreateTablet for tablet 105a52473f7b4917ace1750adea1e147 (DEFAULT_TABLE table=test-table [id=f130d504f5ac4584abeebd79c84f8327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:43.777323 19518 tablet_service.cc:1467] Processing CreateTablet for tablet 105a52473f7b4917ace1750adea1e147 (DEFAULT_TABLE table=test-table [id=f130d504f5ac4584abeebd79c84f8327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:43.777745 19311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105a52473f7b4917ace1750adea1e147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:43.777745 19518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105a52473f7b4917ace1750adea1e147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:43.777711 19080 tablet_service.cc:1467] Processing CreateTablet for tablet 105a52473f7b4917ace1750adea1e147 (DEFAULT_TABLE table=test-table [id=f130d504f5ac4584abeebd79c84f8327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:43.778047 19080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 105a52473f7b4917ace1750adea1e147. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:43.781865 19658 tablet_bootstrap.cc:492] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: Bootstrap starting.
I20260812 01:08:43.782585 19658 tablet_bootstrap.cc:654] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:43.783439 19658 log.cc:824] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:43.783854 19660 tablet_bootstrap.cc:492] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: Bootstrap starting.
I20260812 01:08:43.784597 19658 tablet_bootstrap.cc:492] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: No bootstrap required, opened a new log
I20260812 01:08:43.784618 19660 tablet_bootstrap.cc:654] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:43.784693 19658 ts_tablet_manager.cc:1397] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:08:43.785279 19659 tablet_bootstrap.cc:492] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: Bootstrap starting.
I20260812 01:08:43.785418 19660 log.cc:824] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:43.786209 19659 tablet_bootstrap.cc:654] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:43.786666 19660 tablet_bootstrap.cc:492] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: No bootstrap required, opened a new log
I20260812 01:08:43.786753 19660 ts_tablet_manager.cc:1397] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:08:43.786836 19659 log.cc:824] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:43.787477 19658 raft_consensus.cc:348] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:43.787735 19658 raft_consensus.cc:374] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:43.787784 19658 raft_consensus.cc:729] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 331784e6842746df853c2182331c5d51, State: Initialized, Role: FOLLOWER
I20260812 01:08:43.787904 19659 tablet_bootstrap.cc:492] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: No bootstrap required, opened a new log
I20260812 01:08:43.787935 19658 consensus_queue.cc:260] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:43.788014 19659 ts_tablet_manager.cc:1397] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:08:43.788334 19658 ts_tablet_manager.cc:1428] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:08:43.788478 19202 heartbeater.cc:503] Master 127.17.130.62:44149 was elected leader, sending a full tablet report...
I20260812 01:08:43.789186 19660 raft_consensus.cc:348] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:43.789352 19660 raft_consensus.cc:374] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:43.789412 19660 raft_consensus.cc:729] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76bcffdd5796439898cc4d6d8d79d181, State: Initialized, Role: FOLLOWER
I20260812 01:08:43.789546 19660 consensus_queue.cc:260] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:43.790021 19660 ts_tablet_manager.cc:1428] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 01:08:43.790162 19413 heartbeater.cc:503] Master 127.17.130.62:44149 was elected leader, sending a full tablet report...
I20260812 01:08:43.790459 19659 raft_consensus.cc:348] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:43.790774 19659 raft_consensus.cc:374] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:43.790832 19659 raft_consensus.cc:729] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9071ef9ec0647319a8846dbbdcafc20, State: Initialized, Role: FOLLOWER
I20260812 01:08:43.790964 19659 consensus_queue.cc:260] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:43.791503 19631 heartbeater.cc:503] Master 127.17.130.62:44149 was elected leader, sending a full tablet report...
I20260812 01:08:43.791801 19659 ts_tablet_manager.cc:1428] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
W20260812 01:08:43.803856 19414 tablet.cc:2369] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:08:43.876935 19203 tablet.cc:2369] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:08:43.988600 19632 tablet.cc:2369] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:08:44.058704 19667 raft_consensus.cc:482] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:44.058988 19667 raft_consensus.cc:504] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:44.059444 19667 leader_election.cc:288] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9071ef9ec0647319a8846dbbdcafc20 (127.17.130.3:37845), 331784e6842746df853c2182331c5d51 (127.17.130.1:36697)
I20260812 01:08:44.062427 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.059844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58495 (local address 127.17.130.3:37845)
0812 01:08:44.059898 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:44.059900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:44.059915 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:08:44.059946 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:44.059947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:44.059960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:44.060060 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:44.060210 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.060710 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.061680 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.062073 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.062165 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.062176 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:44.062326 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 01:08:44.062361 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 01:08:44.062427 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.059694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37845 (local address 127.17.130.2:58495)
0812 01:08:44.059800 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:44.059817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:44.059880 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:44.060069 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:44.060070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:44.060072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:44.060135 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:44.060137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.060801 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.060802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:44.061580 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:44.061582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.062164 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.062165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:44.062240 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.062243 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:44.062306 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 01:08:44.062348 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":27}
I20260812 01:08:44.062513 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.059894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:46585 (local address 127.17.130.1:36697)
0812 01:08:44.059941 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:44.059943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:44.060056 (+ 113us) server_negotiation.cc:407] Connection header received
0812 01:08:44.060122 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:44.060123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:44.060135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:44.060208 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:44.060370 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.060829 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.061775 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.062120 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.062203 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.062211 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:44.062379 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 01:08:44.062413 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:08:44.062568 19669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.059822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36697 (local address 127.17.130.2:46585)
0812 01:08:44.059972 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:44.059981 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:44.060063 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:44.060213 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:44.060214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:44.060216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:44.060294 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:44.060301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.060916 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.060917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:44.061680 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:44.061682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.062215 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.062216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:44.062300 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.062304 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:44.062371 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 01:08:44.062411 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260812 01:08:44.062820 19549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a52473f7b4917ace1750adea1e147" candidate_uuid: "76bcffdd5796439898cc4d6d8d79d181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9071ef9ec0647319a8846dbbdcafc20" is_pre_election: true
I20260812 01:08:44.063005 19549 raft_consensus.cc:2454] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bcffdd5796439898cc4d6d8d79d181 in term 0.
I20260812 01:08:44.063311 19230 leader_election.cc:302] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76bcffdd5796439898cc4d6d8d79d181, d9071ef9ec0647319a8846dbbdcafc20; no voters:
I20260812 01:08:44.063467 19667 raft_consensus.cc:2781] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:44.063540 19667 raft_consensus.cc:482] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:44.063477 19118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a52473f7b4917ace1750adea1e147" candidate_uuid: "76bcffdd5796439898cc4d6d8d79d181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331784e6842746df853c2182331c5d51" is_pre_election: true
I20260812 01:08:44.063577 19667 raft_consensus.cc:3037] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:44.063671 19118 raft_consensus.cc:2454] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76bcffdd5796439898cc4d6d8d79d181 in term 0.
I20260812 01:08:44.064409 19667 raft_consensus.cc:504] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:44.064571 19667 leader_election.cc:288] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [CANDIDATE]: Term 1 election: Requested vote from peers d9071ef9ec0647319a8846dbbdcafc20 (127.17.130.3:37845), 331784e6842746df853c2182331c5d51 (127.17.130.1:36697)
I20260812 01:08:44.064764 19549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a52473f7b4917ace1750adea1e147" candidate_uuid: "76bcffdd5796439898cc4d6d8d79d181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9071ef9ec0647319a8846dbbdcafc20"
I20260812 01:08:44.064764 19118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "105a52473f7b4917ace1750adea1e147" candidate_uuid: "76bcffdd5796439898cc4d6d8d79d181" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "331784e6842746df853c2182331c5d51"
I20260812 01:08:44.064854 19118 raft_consensus.cc:3037] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:44.064847 19549 raft_consensus.cc:3037] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:44.065675 19549 raft_consensus.cc:2454] T 105a52473f7b4917ace1750adea1e147 P d9071ef9ec0647319a8846dbbdcafc20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bcffdd5796439898cc4d6d8d79d181 in term 1.
I20260812 01:08:44.065690 19118 raft_consensus.cc:2454] T 105a52473f7b4917ace1750adea1e147 P 331784e6842746df853c2182331c5d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76bcffdd5796439898cc4d6d8d79d181 in term 1.
I20260812 01:08:44.065879 19230 leader_election.cc:302] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 331784e6842746df853c2182331c5d51, 76bcffdd5796439898cc4d6d8d79d181; no voters:
I20260812 01:08:44.065991 19667 raft_consensus.cc:2781] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:44.066169 19667 raft_consensus.cc:686] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [term 1 LEADER]: Becoming Leader. State: Replica: 76bcffdd5796439898cc4d6d8d79d181, State: Running, Role: LEADER
I20260812 01:08:44.066273 19667 consensus_queue.cc:237] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } }
I20260812 01:08:44.067265 18862 catalog_manager.cc:5495] T 105a52473f7b4917ace1750adea1e147 P 76bcffdd5796439898cc4d6d8d79d181 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76bcffdd5796439898cc4d6d8d79d181 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "76bcffdd5796439898cc4d6d8d79d181" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9071ef9ec0647319a8846dbbdcafc20" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "331784e6842746df853c2182331c5d51" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76bcffdd5796439898cc4d6d8d79d181" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38935 } health_report { overall_health: HEALTHY } } }
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:08:44.138280 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.128695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52806 (local address 127.17.130.1:36697)
0812 01:08:44.128798 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:44.128800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:44.128930 (+ 130us) server_negotiation.cc:407] Connection header received
0812 01:08:44.128972 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:44.128973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:44.128985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:44.129141 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:44.129746 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.130247 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.131001 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.131206 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.131274 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.134353 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:44.134368 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:44.134368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:44.134377 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:44.136272 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:44.136954 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:44.136958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:44.136959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:44.137215 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:44.137480 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:44.137481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:44.137481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:44.137643 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:44.137778 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:44.137968 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 01:08:44.138141 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496924, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496924, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:08:44.168955 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.160333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.17.130.1:36697)
0812 01:08:44.160451 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:44.160454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:44.160631 (+ 177us) server_negotiation.cc:407] Connection header received
0812 01:08:44.160738 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:44.160739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:44.160749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:44.160812 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:44.161497 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.162051 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.162958 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.163160 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.163272 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.166196 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:44.166197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:44.166198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:44.166212 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:44.167652 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:44.168035 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:44.168037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:44.168037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:44.168101 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:44.168324 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:44.168325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:44.168325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:44.168434 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:44.168554 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:44.168757 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 01:08:44.168847 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[18785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[18785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:08:44.200413 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:44.190626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.17.130.1:36697)
0812 01:08:44.190706 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:44.190708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:44.190937 (+ 229us) server_negotiation.cc:407] Connection header received
0812 01:08:44.191292 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:44.191293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:44.191305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:44.191368 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:44.192126 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.193006 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.194347 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:44.194536 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:44.194618 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:44.197379 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:44.197381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:44.197381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:44.197389 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:44.199209 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:44.199626 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:44.199628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:44.199628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:44.199690 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:44.199899 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:44.199900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:44.199900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:44.200016 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:44.200099 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:44.200259 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 01:08:44.200329 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 01:08:44.200971 19051 tablet_service.cc:2908] Rejecting scan request for tablet 105a52473f7b4917ace1750adea1e147: Uninitialized: clean time has not yet been initialized
I20260812 01:08:44.202126 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18986
I20260812 01:08:44.215301 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19208
I20260812 01:08:44.229492 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19419
I20260812 01:08:44.243562 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18815
2026-08-12T01:08:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2712 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[19695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[19695](info): set up 2 sockets
Aug 12 01:08:44 dist-test-slave-tfpf krb5kdc[19695](info): commencing operation
krb5kdc: starting...
W20260812 01:08:45.577296 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:08:45 dist-test-slave-tfpf krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:08:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:08:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:08:45.718027 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:41877
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:41877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:45.809724 19711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:45.809962 19711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:45.809986 19711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:45.811153 19711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:08:45.811197 19711 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 01:08:45.811211 19711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:45.811224 19711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:45.811236 19711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:08:45.811249 19711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:08:45.812800 19711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:41877
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:41877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:45.813037 19711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:45.813318 19711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:45.818724 19718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:45.818719 19724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:45.818719 19716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:45.819353 19711 server_base.cc:1034] running on GCE node
I20260812 01:08:45.819779 19711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:45.820123 19711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:45.821307 19711 hybrid_clock.cc:648] HybridClock initialized: now 1786496925821293 us; error 54 us; skew 500 ppm
Aug 12 01:08:45 dist-test-slave-tfpf krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496925, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:45.827881 19711 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:08:45.828387 19711 webserver.cc:460] Webserver started at http://127.17.130.62:34475/ using document root <none> and password file <none>
I20260812 01:08:45.828639 19711 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:45.828713 19711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:45.828878 19711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:45.830090 19711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "84fc952f323541148dd90363500ab3a5"
format_stamp: "Formatted at 2026-08-12 01:08:45 on dist-test-slave-tfpf"
server_key: "4291adb4a204dfccca74b4596b585bca"
server_key_iv: "957aff4130ab180577dcd41e8d1f4f59"
server_key_version: "encryptionkey@0"
I20260812 01:08:45.830487 19711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "84fc952f323541148dd90363500ab3a5"
format_stamp: "Formatted at 2026-08-12 01:08:45 on dist-test-slave-tfpf"
server_key: "4291adb4a204dfccca74b4596b585bca"
server_key_iv: "957aff4130ab180577dcd41e8d1f4f59"
server_key_version: "encryptionkey@0"
I20260812 01:08:45.832419 19711 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:08:45.833408 19741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:45.833606 19711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:45.833729 19711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "84fc952f323541148dd90363500ab3a5"
format_stamp: "Formatted at 2026-08-12 01:08:45 on dist-test-slave-tfpf"
server_key: "4291adb4a204dfccca74b4596b585bca"
server_key_iv: "957aff4130ab180577dcd41e8d1f4f59"
server_key_version: "encryptionkey@0"
I20260812 01:08:45.833812 19711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:45.854861 19711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:45.857616 19711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:45.857817 19711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:45.863631 19711 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:41877
I20260812 01:08:45.863735 19846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:41877 every 8 connection(s)
I20260812 01:08:45.864122 19711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:08:45.865307 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19711
I20260812 01:08:45.865345 19850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:45.865432 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:08:45.865623 17928 external_mini_cluster.cc:1428] Setting key 68bb879e882ef5e6e05e9e73417271e0
I20260812 01:08:45.869966 19850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: Bootstrap starting.
I20260812 01:08:45.870868 19850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:45.871212 19850 log.cc:824] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:45.872474 19850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: No bootstrap required, opened a new log
Aug 12 01:08:45 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:45.875603 19850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc952f323541148dd90363500ab3a5" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41877 } }
I20260812 01:08:45.875841 19850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:45.875888 19850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84fc952f323541148dd90363500ab3a5, State: Initialized, Role: FOLLOWER
I20260812 01:08:45.876075 19850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc952f323541148dd90363500ab3a5" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41877 } }
I20260812 01:08:45.876173 19850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:08:45.876209 19850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:08:45.876263 19850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:45.878427 19850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc952f323541148dd90363500ab3a5" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41877 } }
I20260812 01:08:45.878693 19850 leader_election.cc:302] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84fc952f323541148dd90363500ab3a5; no voters:
I20260812 01:08:45.878985 19850 leader_election.cc:288] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:08:45.879097 19856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:45.879298 19850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:08:45.879405 19856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [term 1 LEADER]: Becoming Leader. State: Replica: 84fc952f323541148dd90363500ab3a5, State: Running, Role: LEADER
I20260812 01:08:45.879580 19856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc952f323541148dd90363500ab3a5" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41877 } }
I20260812 01:08:45.880018 19857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84fc952f323541148dd90363500ab3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc952f323541148dd90363500ab3a5" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41877 } } }
I20260812 01:08:45.880131 19857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:45.880313 19860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84fc952f323541148dd90363500ab3a5. Latest consensus state: current_term: 1 leader_uuid: "84fc952f323541148dd90363500ab3a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84fc952f323541148dd90363500ab3a5" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41877 } } }
I20260812 01:08:45.880390 19860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5 [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:45.880842 19866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:08:45.881212 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:45.866576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.17.130.62:41877)
0812 01:08:45.866811 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:45.866826 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:45.866864 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:08:45.867629 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:45.867638 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:45.867717 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:45.867879 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:45.868877 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:45.869928 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:45.871042 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:45.871587 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:45.871707 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:45.874549 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:45.874555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:45.874556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:45.874571 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:45.877482 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:45.878213 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:45.878217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:45.878218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:45.878357 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:45.878702 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:45.878704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:45.878704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:45.879161 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:45.880399 (+ 1238us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:45.880554 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 01:08:45.880922 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":118,"threads_started":1}
I20260812 01:08:45.881460 19866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:08:45.883534 19866 catalog_manager.cc:1347] Generated new cluster ID: 4f54d1f09e864d61962bda0cfe01fdbf
I20260812 01:08:45.883610 19866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:08:45.908779 19866 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:08:45.909641 19866 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:08:45.914072 19866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: Generated new TSK 0
I20260812 01:08:45.914275 19866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:08:45.949142 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41877
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:46.043817 19895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:46.044076 19895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:46.044131 19895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:46.045545 19895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:46.045627 19895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:46.045666 19895 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:08:46.045724 19895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:08:46.047366 19895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:41877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:46.047695 19895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:46.047982 19895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:46.053936 19910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:46.053985 19909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:46.054049 19915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:46.054075 19895 server_base.cc:1034] running on GCE node
I20260812 01:08:46.054348 19895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:46.054647 19895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:46.055819 19895 hybrid_clock.cc:648] HybridClock initialized: now 1786496926055792 us; error 50 us; skew 500 ppm
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:46.058723 19895 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:08:46.059279 19895 webserver.cc:460] Webserver started at http://127.17.130.1:44233/ using document root <none> and password file <none>
I20260812 01:08:46.059553 19895 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:46.059626 19895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:46.059854 19895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:46.061025 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "a338c1ab4a474ebe874488b6eba83c88"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "da3b097996691f0390293c680a29fd41"
server_key_iv: "621b7d3ca6c34d52597d8512d2f236a1"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.061522 19895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "a338c1ab4a474ebe874488b6eba83c88"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "da3b097996691f0390293c680a29fd41"
server_key_iv: "621b7d3ca6c34d52597d8512d2f236a1"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.063028 19895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:08:46.063900 19929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.064198 19895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:46.064337 19895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "a338c1ab4a474ebe874488b6eba83c88"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "da3b097996691f0390293c680a29fd41"
server_key_iv: "621b7d3ca6c34d52597d8512d2f236a1"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.064419 19895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:46.078939 19895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:46.082007 19895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:46.082223 19895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:46.082995 19895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:46.083055 19895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.083086 19895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:46.083106 19895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:46.091266 19895 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:36127
I20260812 01:08:46.091724 19895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:08:46.092519 20105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:36127 every 8 connection(s)
I20260812 01:08:46.095274 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.083584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:50809 (local address 127.17.130.62:41877)
0812 01:08:46.083672 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.083675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.083884 (+ 209us) server_negotiation.cc:407] Connection header received
0812 01:08:46.084393 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.084394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.084408 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.084495 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.085434 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.086298 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.087430 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.087655 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.087738 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.090587 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.090590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.090590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.090599 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.093274 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.093899 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.093901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.093901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.093981 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.094421 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.094423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.094424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.094605 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.094702 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.095022 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.095178 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 01:08:46.095467 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.083538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.17.130.1:50809)
0812 01:08:46.083738 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.083782 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.084292 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.084587 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.084590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.085010 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.085313 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.085319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.086413 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.086414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.087319 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.087322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.087765 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.087766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.087904 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.088673 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.088730 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.090465 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.093425 (+ 2960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.093437 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.093441 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.093792 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.094069 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.094071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.094071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.094207 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.094698 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.094709 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.094869 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.094996 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.095193 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260812 01:08:46.096087 20106 heartbeater.cc:347] Connected to a master server at 127.17.130.62:41877
I20260812 01:08:46.096194 20106 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:46.096406 20106 heartbeater.cc:511] Master 127.17.130.62:41877 requested a full tablet report, sending...
I20260812 01:08:46.096774 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19895
I20260812 01:08:46.096899 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:08:46.097052 19764 ts_manager.cc:194] Registered new tserver with Master: a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1:36127)
I20260812 01:08:46.097141 17928 external_mini_cluster.cc:1428] Setting key f0112353bc433529ba0316422003d76b
I20260812 01:08:46.098110 19764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:50809
I20260812 01:08:46.104691 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.097535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:60917 (local address 127.17.130.62:41877)
0812 01:08:46.097621 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.097623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.097687 (+ 64us) server_negotiation.cc:407] Connection header received
0812 01:08:46.097763 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.097763 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.097773 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.097876 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.098325 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.099051 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.100227 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.100434 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.100524 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.101629 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.101630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.101631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.101639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.103478 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.103850 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.103851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.103852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.103914 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.104125 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.104126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.104126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.104249 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.104325 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.104494 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.104585 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 01:08:46.104703 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.097462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.17.130.1:60917)
0812 01:08:46.097633 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.097644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.097704 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.097884 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.097886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.098149 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.098242 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.098245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.099168 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.099169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.100142 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.100145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.100528 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.100529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.100611 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.101176 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.101182 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.101522 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.103580 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.103581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.103582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.103768 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.103992 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.103993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.103993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.104060 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.104329 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.104330 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.104381 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.104466 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.104592 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:08:46.126333 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41877
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:46.219944 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:46.220228 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:46.220275 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:46.221755 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:46.221856 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:46.221897 20122 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:08:46.221942 20122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:08:46.223687 20122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:41877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:46.224022 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:46.224294 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:46.230309 20131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:46.230312 20129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:46.230309 20136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:46.230533 20122 server_base.cc:1034] running on GCE node
I20260812 01:08:46.230929 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:46.231256 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:46.232458 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786496926232446 us; error 35 us; skew 500 ppm
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:46.235832 20122 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:08:46.236465 20122 webserver.cc:460] Webserver started at http://127.17.130.2:43959/ using document root <none> and password file <none>
I20260812 01:08:46.236750 20122 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:46.236827 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:46.236994 20122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:46.238444 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "e2957c9251cd4927ba0eea3bcdb0cf62"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "edae512fa7729d30643dcf90fe90c1de"
server_key_iv: "6bd1382d904651c73ef63d97c9fa28b6"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.238966 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "e2957c9251cd4927ba0eea3bcdb0cf62"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "edae512fa7729d30643dcf90fe90c1de"
server_key_iv: "6bd1382d904651c73ef63d97c9fa28b6"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.240587 20122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:08:46.242035 20150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.242504 20122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:46.242852 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "e2957c9251cd4927ba0eea3bcdb0cf62"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "edae512fa7729d30643dcf90fe90c1de"
server_key_iv: "6bd1382d904651c73ef63d97c9fa28b6"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.242981 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:46.257470 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:46.260440 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:46.260656 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:46.261485 20122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:46.261564 20122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.261603 20122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:46.261644 20122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:46.269778 20122 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:40487
I20260812 01:08:46.269912 20325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:40487 every 8 connection(s)
I20260812 01:08:46.270282 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:08:46.272716 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.261895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:40811 (local address 127.17.130.62:41877)
0812 01:08:46.261983 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.261986 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.262267 (+ 281us) server_negotiation.cc:407] Connection header received
0812 01:08:46.263069 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.263071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.263084 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.263185 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.263888 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.264414 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.265538 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.265744 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.265857 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.269396 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.269397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.269398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.269407 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.271206 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.271674 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.271675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.271676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.271746 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.272009 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.272010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.272011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.272153 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.272235 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.272526 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.272615 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:08:46.272953 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.261887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.17.130.2:40811)
0812 01:08:46.262090 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.262138 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.262912 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.263284 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.263288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.263561 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.263785 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.263789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.264522 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.264522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.265441 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.265444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.265871 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.265872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.266003 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.266610 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.266620 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.269272 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.271325 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.271333 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.271335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.271587 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.271819 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.271820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.271820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.271935 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.272226 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.272234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.272384 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.272500 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.272686 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260812 01:08:46.273146 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20122
I20260812 01:08:46.273242 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:08:46.273480 17928 external_mini_cluster.cc:1428] Setting key c7847b058d58b71a4e17e5bad4baebf4
I20260812 01:08:46.273502 20328 heartbeater.cc:347] Connected to a master server at 127.17.130.62:41877
I20260812 01:08:46.273593 20328 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:46.273808 20328 heartbeater.cc:511] Master 127.17.130.62:41877 requested a full tablet report, sending...
I20260812 01:08:46.274148 19764 ts_manager.cc:194] Registered new tserver with Master: e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487)
I20260812 01:08:46.274637 19764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:40811
I20260812 01:08:46.283684 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.275232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:41547 (local address 127.17.130.62:41877)
0812 01:08:46.275275 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.275277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.276217 (+ 940us) server_negotiation.cc:407] Connection header received
0812 01:08:46.276249 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.276250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.276263 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.276344 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.276653 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.277454 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.278561 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.278748 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.278814 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.280133 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.280134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.280134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.280141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.282180 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.282646 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.282648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.282648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.282711 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.282999 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.283000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.283001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.283189 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.283259 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.283504 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.283599 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:08:46.283705 20337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.275148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.17.130.2:41547)
0812 01:08:46.276131 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.276143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.276202 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.276348 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.276348 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.276514 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.276578 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.276580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.277576 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.277577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.278452 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.278456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.278835 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.278836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.278921 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.279529 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.279535 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.280016 (+ 481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.282275 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.282277 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.282277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.282535 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.282795 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.282796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.282796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.282878 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.283276 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.283277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.283342 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.283427 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.283570 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:08:46.306229 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41877
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:46.397313 20341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:46.397548 20341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:46.397578 20341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:46.398756 20341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:46.398794 20341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:46.398815 20341 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:08:46.398833 20341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:08:46.400513 20341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:41877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:46.400740 20341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:46.400945 20341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:46.408344 20351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:46.408383 20341 server_base.cc:1034] running on GCE node
W20260812 01:08:46.408286 20349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:46.408615 20355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:46.408845 20341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:46.409204 20341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:46.410403 20341 hybrid_clock.cc:648] HybridClock initialized: now 1786496926410387 us; error 50 us; skew 500 ppm
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:46.413574 20341 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:08:46.414163 20341 webserver.cc:460] Webserver started at http://127.17.130.3:34073/ using document root <none> and password file <none>
I20260812 01:08:46.414402 20341 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:46.414448 20341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:46.414600 20341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:46.415838 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "a2756137813e43b78c4b04cc26cf53f8"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "0ada10949946d2652ffeed2b0a0d1ae0"
server_key_iv: "93162ac97e55b30d52ff031f9571296f"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.416230 20341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "a2756137813e43b78c4b04cc26cf53f8"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "0ada10949946d2652ffeed2b0a0d1ae0"
server_key_iv: "93162ac97e55b30d52ff031f9571296f"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.417853 20341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:08:46.418781 20364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.419147 20341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:46.419273 20341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "a2756137813e43b78c4b04cc26cf53f8"
format_stamp: "Formatted at 2026-08-12 01:08:46 on dist-test-slave-tfpf"
server_key: "0ada10949946d2652ffeed2b0a0d1ae0"
server_key_iv: "93162ac97e55b30d52ff031f9571296f"
server_key_version: "encryptionkey@0"
I20260812 01:08:46.419358 20341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:46.447139 20341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:46.449566 20341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:46.449723 20341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:46.450394 20341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:46.450433 20341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.450461 20341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:46.450476 20341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:46.459281 20341 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:44893
I20260812 01:08:46.459342 20545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:44893 every 8 connection(s)
I20260812 01:08:46.459743 20341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:08:46.462014 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.451079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:40467 (local address 127.17.130.62:41877)
0812 01:08:46.451143 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.451145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.451340 (+ 195us) server_negotiation.cc:407] Connection header received
0812 01:08:46.451960 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.451962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.451975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.452030 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.453129 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.453761 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.454953 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.455229 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.455320 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.458580 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.458582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.458582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.458591 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.460361 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.460771 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.460772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.460772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.460845 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.461300 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.461301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.461301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.461450 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.461527 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.461860 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.461946 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:08:46.462167 20378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.451008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.17.130.3:40467)
0812 01:08:46.451218 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.451255 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.451842 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.452117 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.452121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.452514 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.452808 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.452815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.453865 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.453866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.454822 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.454826 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.455335 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.455337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.455519 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.456166 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.456179 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.458457 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.460443 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.460448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.460450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.460676 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.460924 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.460924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.460925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.461206 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.461558 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.461565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.461696 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.461801 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.461953 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":112,"threads_started":1}
I20260812 01:08:46.462674 20546 heartbeater.cc:347] Connected to a master server at 127.17.130.62:41877
I20260812 01:08:46.462764 20546 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:46.462967 20546 heartbeater.cc:511] Master 127.17.130.62:41877 requested a full tablet report, sending...
I20260812 01:08:46.463451 19764 ts_manager.cc:194] Registered new tserver with Master: a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893)
I20260812 01:08:46.463769 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20341
I20260812 01:08:46.463891 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:08:46.463979 19764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:40467
I20260812 01:08:46.464099 17928 external_mini_cluster.cc:1428] Setting key 20f03abeb36cf84f05d4c701202730ca
I20260812 01:08:46.465003 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:08:46.471297 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.464006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:47789 (local address 127.17.130.62:41877)
0812 01:08:46.464062 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.464064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.464417 (+ 353us) server_negotiation.cc:407] Connection header received
0812 01:08:46.464523 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.464524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.464542 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.464605 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.464973 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.465544 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.466618 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.466890 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.466990 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.468478 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.468480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.468481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.468490 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.470070 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.470471 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.470472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.470473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.470534 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.470743 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.470743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.470744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.470868 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.470948 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.471112 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.471198 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:08:46.471349 20554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.464076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.17.130.3:47789)
0812 01:08:46.464358 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.464371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.464435 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.464622 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.464623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.464820 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.464899 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.464901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.465645 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.465646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.466392 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.466400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.466992 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.466994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.467102 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.467918 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.467924 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.468335 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.470191 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.470193 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.470193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.470379 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.470604 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.470605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.470606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.470673 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.470939 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.470940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.470993 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.471082 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.471245 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:46.494565 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.485260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32798 (local address 127.17.130.62:41877)
0812 01:08:46.485330 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.485332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.485341 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:08:46.485420 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.485421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.485431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.485492 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.486191 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.486990 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.488561 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.488954 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.489031 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.491713 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.491715 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.491715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.491724 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.493307 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.493694 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.493695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.493696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.493755 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.493985 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.493986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.493986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.494115 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.494228 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.494362 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.494451 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:08:46.496665 19764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:08:46.499503 19764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:46.507414 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.502873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:44893 (local address 127.0.0.1:41082)
0812 01:08:46.503299 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.503313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.503511 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.503859 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.503863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.503865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.503978 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.503981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.505008 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.505009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.506193 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.506197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.507001 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.507003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.507095 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.507112 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.507226 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.507293 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":134,"threads_started":1}
I20260812 01:08:46.507431 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.502992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41082 (local address 127.17.130.3:44893)
0812 01:08:46.503274 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.503280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.503589 (+ 309us) server_negotiation.cc:407] Connection header received
0812 01:08:46.503648 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.503650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.503669 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.503757 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.504086 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.504876 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.506303 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.506870 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.506996 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.507072 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.507247 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.507291 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":118,"threads_started":1}
I20260812 01:08:46.508205 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.503134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36127 (local address 127.0.0.1:58962)
0812 01:08:46.503472 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.503482 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.503630 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.504009 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.504010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.504012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.504070 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.504072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.505588 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.505589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.506873 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.506876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.507951 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.507953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.508020 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.508024 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.508079 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.508121 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":34,"thread_start_us":115,"threads_started":1}
I20260812 01:08:46.508500 20579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.502810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40487 (local address 127.0.0.1:34368)
0812 01:08:46.503055 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.503098 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.503349 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.504072 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.504073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.504075 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.504141 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.504142 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.505593 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.505594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.506873 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.506876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.508282 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.508284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.508357 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.508362 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.508396 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.508434 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":87,"thread_start_us":88,"threads_started":1}
I20260812 01:08:46.508358 20431 tablet_service.cc:1467] Processing CreateTablet for tablet 47b39a36965b4df2aef3074b0d3e18a2 (DEFAULT_TABLE table=test-table [id=a6a0ddffddb443429f6992a4aa84a374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:46.508575 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.503440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.17.130.1:36127)
0812 01:08:46.503716 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.503720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.503732 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:08:46.503776 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.503779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.503794 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.504005 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.504143 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.505481 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.506993 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.507782 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.507856 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.507910 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.508393 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.508438 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":152,"threads_started":1}
I20260812 01:08:46.508877 20431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b39a36965b4df2aef3074b0d3e18a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.509254 20006 tablet_service.cc:1467] Processing CreateTablet for tablet 47b39a36965b4df2aef3074b0d3e18a2 (DEFAULT_TABLE table=test-table [id=a6a0ddffddb443429f6992a4aa84a374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:46.509627 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b39a36965b4df2aef3074b0d3e18a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.512521 20587 tablet_bootstrap.cc:492] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Bootstrap starting.
I20260812 01:08:46.513341 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.502935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.17.130.2:40487)
0812 01:08:46.503754 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.503758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.503768 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:08:46.503805 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.503807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.503824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.504224 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.504285 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.505483 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.506995 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.508181 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.508695 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.508763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.508789 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.513217 (+ 4428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":105,"threads_started":1}
I20260812 01:08:46.513415 20590 tablet_bootstrap.cc:492] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Bootstrap starting.
I20260812 01:08:46.513639 20587 tablet_bootstrap.cc:654] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.513998 20587 log.cc:824] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:46.514217 20590 tablet_bootstrap.cc:654] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.514386 20227 tablet_service.cc:1467] Processing CreateTablet for tablet 47b39a36965b4df2aef3074b0d3e18a2 (DEFAULT_TABLE table=test-table [id=a6a0ddffddb443429f6992a4aa84a374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:46.514621 20590 log.cc:824] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:46.514793 20227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47b39a36965b4df2aef3074b0d3e18a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.514851 20587 tablet_bootstrap.cc:492] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: No bootstrap required, opened a new log
I20260812 01:08:46.514925 20587 ts_tablet_manager.cc:1397] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:08:46.516352 20587 raft_consensus.cc:348] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.516479 20587 raft_consensus.cc:374] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.516501 20587 raft_consensus.cc:729] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2756137813e43b78c4b04cc26cf53f8, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.516600 20587 consensus_queue.cc:260] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.516829 20587 ts_tablet_manager.cc:1428] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:08:46.516949 20546 heartbeater.cc:503] Master 127.17.130.62:41877 was elected leader, sending a full tablet report...
I20260812 01:08:46.517539 20590 tablet_bootstrap.cc:492] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: No bootstrap required, opened a new log
I20260812 01:08:46.517634 20590 ts_tablet_manager.cc:1397] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:08:46.518357 20602 tablet_bootstrap.cc:492] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Bootstrap starting.
I20260812 01:08:46.519212 20602 tablet_bootstrap.cc:654] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.519572 20602 log.cc:824] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:46.519575 20590 raft_consensus.cc:348] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.519721 20590 raft_consensus.cc:374] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.519748 20590 raft_consensus.cc:729] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a338c1ab4a474ebe874488b6eba83c88, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.519831 20590 consensus_queue.cc:260] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.520174 20590 ts_tablet_manager.cc:1428] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:08:46.520267 20106 heartbeater.cc:503] Master 127.17.130.62:41877 was elected leader, sending a full tablet report...
I20260812 01:08:46.520679 20602 tablet_bootstrap.cc:492] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: No bootstrap required, opened a new log
I20260812 01:08:46.520789 20602 ts_tablet_manager.cc:1397] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:08:46.522724 20602 raft_consensus.cc:348] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.522899 20602 raft_consensus.cc:374] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.522948 20602 raft_consensus.cc:729] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2957c9251cd4927ba0eea3bcdb0cf62, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.523092 20602 consensus_queue.cc:260] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.523429 20602 ts_tablet_manager.cc:1428] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:08:46.523456 20328 heartbeater.cc:503] Master 127.17.130.62:41877 was elected leader, sending a full tablet report...
W20260812 01:08:46.593325 20109 tablet.cc:2369] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:08:46.632601 20604 raft_consensus.cc:482] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:46.632750 20604 raft_consensus.cc:504] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.633188 20604 leader_election.cc:288] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893), e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487)
I20260812 01:08:46.636353 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.633546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:34063 (local address 127.17.130.3:44893)
0812 01:08:46.633609 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.633611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.633672 (+ 61us) server_negotiation.cc:407] Connection header received
0812 01:08:46.633749 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.633749 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.633759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.633837 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.633998 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.634483 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.635606 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.635977 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.636064 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.636075 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.636237 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.636276 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:08:46.636358 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.633411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:44893 (local address 127.17.130.1:34063)
0812 01:08:46.633604 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.633621 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.633691 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.633844 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.633845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.633847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.633919 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.633923 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.634623 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.634624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.635503 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.635505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.636083 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.636084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.636162 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.636166 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.636232 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.636275 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260812 01:08:46.636443 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.633555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:48793 (local address 127.17.130.2:40487)
0812 01:08:46.633609 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.633612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.633793 (+ 181us) server_negotiation.cc:407] Connection header received
0812 01:08:46.633872 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.633873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.633884 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.633964 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.634145 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.634656 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.635637 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.636042 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.636133 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.636142 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.636325 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.636364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:08:46.636507 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.633451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40487 (local address 127.17.130.1:48793)
0812 01:08:46.633711 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.633722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.633808 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.633970 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.633971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.633973 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.634058 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.634060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.634758 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.634759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.635528 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.635530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.636133 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.636134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.636225 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.636229 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.636315 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.636365 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":75,"threads_started":1}
I20260812 01:08:46.636893 20259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b39a36965b4df2aef3074b0d3e18a2" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" is_pre_election: true
I20260812 01:08:46.637079 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b39a36965b4df2aef3074b0d3e18a2" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2756137813e43b78c4b04cc26cf53f8" is_pre_election: true
I20260812 01:08:46.637151 20259 raft_consensus.cc:2454] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 0.
I20260812 01:08:46.637266 20476 raft_consensus.cc:2454] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 0.
I20260812 01:08:46.637399 19934 leader_election.cc:302] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a338c1ab4a474ebe874488b6eba83c88, e2957c9251cd4927ba0eea3bcdb0cf62; no voters:
I20260812 01:08:46.637554 20604 raft_consensus.cc:2781] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:46.637632 20604 raft_consensus.cc:482] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:46.637652 20604 raft_consensus.cc:3037] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.638536 20604 raft_consensus.cc:504] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.638772 20604 leader_election.cc:288] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 election: Requested vote from peers a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893), e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487)
I20260812 01:08:46.638953 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b39a36965b4df2aef3074b0d3e18a2" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2756137813e43b78c4b04cc26cf53f8"
I20260812 01:08:46.639019 20476 raft_consensus.cc:3037] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.639106 20259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47b39a36965b4df2aef3074b0d3e18a2" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62"
I20260812 01:08:46.639218 20259 raft_consensus.cc:3037] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.640009 20476 raft_consensus.cc:2454] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 1.
I20260812 01:08:46.640111 20259 raft_consensus.cc:2454] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 1.
I20260812 01:08:46.640242 19936 leader_election.cc:302] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2756137813e43b78c4b04cc26cf53f8, a338c1ab4a474ebe874488b6eba83c88; no voters:
I20260812 01:08:46.640388 20604 raft_consensus.cc:2781] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:46.640542 20604 raft_consensus.cc:686] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 1 LEADER]: Becoming Leader. State: Replica: a338c1ab4a474ebe874488b6eba83c88, State: Running, Role: LEADER
I20260812 01:08:46.640646 20604 consensus_queue.cc:237] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } }
I20260812 01:08:46.641741 19764 catalog_manager.cc:5495] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 reported cstate change: term changed from 0 to 1, leader changed from <none> to a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "a338c1ab4a474ebe874488b6eba83c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } health_report { overall_health: UNKNOWN } } }
I20260812 01:08:46.656287 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.653395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.17.130.1:36127)
0812 01:08:46.653461 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.653464 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.653476 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:08:46.653585 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.653587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.653609 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.653719 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:46.653906 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.654438 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.655335 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.655566 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.655653 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.655886 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.655999 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.656099 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.656163 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.656191 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:08:46.658550 20259 raft_consensus.cc:1261] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Refusing update from remote peer a338c1ab4a474ebe874488b6eba83c88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:46.658550 20476 raft_consensus.cc:1261] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Refusing update from remote peer a338c1ab4a474ebe874488b6eba83c88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:46.658948 20604 consensus_queue.cc:1035] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:46.659133 20610 consensus_queue.cc:1035] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:46.662041 20615 mvcc.cc:205] Tried to move back new op lower bound from 7317491411590320128 to 7317491411520839680. Current Snapshot: MvccSnapshot[applied={T|T < 7317491411590320128}]
I20260812 01:08:46.663133 20616 mvcc.cc:205] Tried to move back new op lower bound from 7317491411590320128 to 7317491411520839680. Current Snapshot: MvccSnapshot[applied={T|T < 7317491411590320128}]
I20260812 01:08:46.666008 19767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:46.666146 19767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:46.666972 19767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: Sending DeleteTablet for 3 replicas of tablet 47b39a36965b4df2aef3074b0d3e18a2
I20260812 01:08:46.667460 20431 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b39a36965b4df2aef3074b0d3e18a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:46 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:41082
I20260812 01:08:46.667490 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b39a36965b4df2aef3074b0d3e18a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:46 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:58962
I20260812 01:08:46.667752 20626 tablet_replica.cc:330] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: stopping tablet replica
I20260812 01:08:46.667752 20227 tablet_service.cc:1514] Processing DeleteTablet for tablet 47b39a36965b4df2aef3074b0d3e18a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:46 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:34368
I20260812 01:08:46.667863 20626 raft_consensus.cc:2229] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:08:46.667871 19767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 01:08:46.667964 20628 tablet_replica.cc:330] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: stopping tablet replica
I20260812 01:08:46.667979 20626 raft_consensus.cc:2258] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 01:08:46.668025 19767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:46.668054 20628 raft_consensus.cc:2229] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:46.668164 20628 raft_consensus.cc:2258] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:46.669034 20626 ts_tablet_manager.cc:1905] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:46.669498 20006 tablet_service.cc:1467] Processing CreateTablet for tablet 197b4dccebf54573b340609b76d35d07 (DEFAULT_TABLE table=test-table [id=ea88f4a65be44ce7913a20efde903d15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:46.669562 20431 tablet_service.cc:1467] Processing CreateTablet for tablet 197b4dccebf54573b340609b76d35d07 (DEFAULT_TABLE table=test-table [id=ea88f4a65be44ce7913a20efde903d15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:46.669638 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197b4dccebf54573b340609b76d35d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.669683 20431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197b4dccebf54573b340609b76d35d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.671100 20626 ts_tablet_manager.cc:1918] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:08:46.671247 20626 log.cc:1196] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/47b39a36965b4df2aef3074b0d3e18a2
I20260812 01:08:46.671615 20626 ts_tablet_manager.cc:1939] T 47b39a36965b4df2aef3074b0d3e18a2 P a338c1ab4a474ebe874488b6eba83c88: Deleting consensus metadata
I20260812 01:08:46.671675 20227 tablet_service.cc:1467] Processing CreateTablet for tablet 197b4dccebf54573b340609b76d35d07 (DEFAULT_TABLE table=test-table [id=ea88f4a65be44ce7913a20efde903d15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:46.671789 20587 tablet_bootstrap.cc:492] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Bootstrap starting.
I20260812 01:08:46.671816 20227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197b4dccebf54573b340609b76d35d07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.672266 20587 tablet_bootstrap.cc:654] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.673195 20627 tablet_replica.cc:330] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: stopping tablet replica
I20260812 01:08:46.673309 20587 tablet_bootstrap.cc:492] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: No bootstrap required, opened a new log
I20260812 01:08:46.673338 20627 raft_consensus.cc:2229] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:46.673369 20587 ts_tablet_manager.cc:1397] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:08:46.673462 20627 raft_consensus.cc:2258] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:46.673539 20587 raft_consensus.cc:348] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.673640 20587 raft_consensus.cc:374] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.673646 20590 tablet_bootstrap.cc:492] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Bootstrap starting.
I20260812 01:08:46.673676 20587 raft_consensus.cc:729] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2756137813e43b78c4b04cc26cf53f8, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.673722 20587 consensus_queue.cc:260] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.673834 20587 ts_tablet_manager.cc:1428] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:08:46.674103 20590 tablet_bootstrap.cc:654] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.674698 20602 tablet_bootstrap.cc:492] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Bootstrap starting.
I20260812 01:08:46.675040 20590 tablet_bootstrap.cc:492] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: No bootstrap required, opened a new log
I20260812 01:08:46.675102 20590 ts_tablet_manager.cc:1397] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:46.675231 20602 tablet_bootstrap.cc:654] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.675251 20590 raft_consensus.cc:348] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.675338 20590 raft_consensus.cc:374] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.675364 20590 raft_consensus.cc:729] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a338c1ab4a474ebe874488b6eba83c88, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.675406 20590 consensus_queue.cc:260] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.675508 20590 ts_tablet_manager.cc:1428] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:46.675815 19748 catalog_manager.cc:4841] TS a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1:36127): tablet 47b39a36965b4df2aef3074b0d3e18a2 (table test-table [id=a6a0ddffddb443429f6992a4aa84a374]) successfully deleted
I20260812 01:08:46.675879 20627 ts_tablet_manager.cc:1905] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:46.676285 20602 tablet_bootstrap.cc:492] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: No bootstrap required, opened a new log
I20260812 01:08:46.676339 20602 ts_tablet_manager.cc:1397] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:08:46.676614 20602 raft_consensus.cc:348] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.676733 20602 raft_consensus.cc:374] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.676760 20602 raft_consensus.cc:729] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2957c9251cd4927ba0eea3bcdb0cf62, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.676813 20602 consensus_queue.cc:260] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.676925 20602 ts_tablet_manager.cc:1428] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:46.677443 20628 ts_tablet_manager.cc:1905] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:46.677968 20627 ts_tablet_manager.cc:1918] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:08:46.678054 20627 log.cc:1196] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/47b39a36965b4df2aef3074b0d3e18a2
I20260812 01:08:46.678341 20627 ts_tablet_manager.cc:1939] T 47b39a36965b4df2aef3074b0d3e18a2 P a2756137813e43b78c4b04cc26cf53f8: Deleting consensus metadata
I20260812 01:08:46.678884 19748 catalog_manager.cc:4841] TS a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893): tablet 47b39a36965b4df2aef3074b0d3e18a2 (table test-table [id=a6a0ddffddb443429f6992a4aa84a374]) successfully deleted
I20260812 01:08:46.679376 20628 ts_tablet_manager.cc:1918] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:08:46.679461 20628 log.cc:1196] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/47b39a36965b4df2aef3074b0d3e18a2
I20260812 01:08:46.679744 20628 ts_tablet_manager.cc:1939] T 47b39a36965b4df2aef3074b0d3e18a2 P e2957c9251cd4927ba0eea3bcdb0cf62: Deleting consensus metadata
I20260812 01:08:46.680271 19747 catalog_manager.cc:4841] TS e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487): tablet 47b39a36965b4df2aef3074b0d3e18a2 (table test-table [id=a6a0ddffddb443429f6992a4aa84a374]) successfully deleted
W20260812 01:08:46.710496 20547 tablet.cc:2369] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:08:46.771160 20329 tablet.cc:2369] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:08:46.794091 20610 raft_consensus.cc:482] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:46.794209 20610 raft_consensus.cc:504] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.794384 20610 leader_election.cc:288] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487), a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893)
I20260812 01:08:46.794633 20259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197b4dccebf54573b340609b76d35d07" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" is_pre_election: true
I20260812 01:08:46.794632 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197b4dccebf54573b340609b76d35d07" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2756137813e43b78c4b04cc26cf53f8" is_pre_election: true
I20260812 01:08:46.794734 20476 raft_consensus.cc:2454] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 0.
I20260812 01:08:46.794734 20259 raft_consensus.cc:2454] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 0.
I20260812 01:08:46.794899 19934 leader_election.cc:302] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a338c1ab4a474ebe874488b6eba83c88, e2957c9251cd4927ba0eea3bcdb0cf62; no voters:
I20260812 01:08:46.795027 20610 raft_consensus.cc:2781] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:46.795091 20610 raft_consensus.cc:482] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:46.795107 20610 raft_consensus.cc:3037] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.795868 20610 raft_consensus.cc:504] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.796139 20610 leader_election.cc:288] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 election: Requested vote from peers e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487), a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893)
I20260812 01:08:46.796310 20259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197b4dccebf54573b340609b76d35d07" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62"
I20260812 01:08:46.796438 20259 raft_consensus.cc:3037] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.796386 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197b4dccebf54573b340609b76d35d07" candidate_uuid: "a338c1ab4a474ebe874488b6eba83c88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2756137813e43b78c4b04cc26cf53f8"
I20260812 01:08:46.796491 20476 raft_consensus.cc:3037] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.797179 20259 raft_consensus.cc:2454] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 1.
I20260812 01:08:46.797305 20476 raft_consensus.cc:2454] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a338c1ab4a474ebe874488b6eba83c88 in term 1.
I20260812 01:08:46.797410 19934 leader_election.cc:302] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a338c1ab4a474ebe874488b6eba83c88, e2957c9251cd4927ba0eea3bcdb0cf62; no voters:
I20260812 01:08:46.797518 20610 raft_consensus.cc:2781] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:46.797590 20610 raft_consensus.cc:686] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 1 LEADER]: Becoming Leader. State: Replica: a338c1ab4a474ebe874488b6eba83c88, State: Running, Role: LEADER
I20260812 01:08:46.797662 20610 consensus_queue.cc:237] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.798451 19767 catalog_manager.cc:5495] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 reported cstate change: term changed from 0 to 1, leader changed from <none> to a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "a338c1ab4a474ebe874488b6eba83c88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } health_report { overall_health: UNKNOWN } } }
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496925, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:46.869773 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.861913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41877 (local address 127.0.0.1:32800)
0812 01:08:46.862128 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.862137 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.862193 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.862370 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.862371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.862684 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:46.862733 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.862735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.863554 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.863556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.864099 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.864101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.864563 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.864564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.864612 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.865245 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:46.865250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:46.866625 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:46.868414 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.868422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.868425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.868684 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.868920 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:46.868921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:46.868921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:46.869117 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:46.869423 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:46.869431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:46.869498 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.869572 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.869683 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":119,"threads_started":1}
I20260812 01:08:46.869879 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.861981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.17.130.62:41877)
0812 01:08:46.862071 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.862073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.862249 (+ 176us) server_negotiation.cc:407] Connection header received
0812 01:08:46.862285 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.862286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.862298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.862351 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.862809 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.863394 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.864207 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.864477 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.864555 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.866728 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:46.866729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:46.866730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:46.866737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:46.868314 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.868771 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.868773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.868773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.868843 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:46.869201 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:46.869202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:46.869202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:46.869327 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:46.869392 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.869605 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.869751 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":60}
I20260812 01:08:46.872730 19767 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:32800:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 01:08:46.873030 19767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:46.874969 20227 tablet_service.cc:1467] Processing CreateTablet for tablet e27ed649489145718ab95a487a880c22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df69ffb38a98489580e5734d79c69b05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:08:46.875016 20006 tablet_service.cc:1467] Processing CreateTablet for tablet e27ed649489145718ab95a487a880c22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df69ffb38a98489580e5734d79c69b05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:08:46.875094 20431 tablet_service.cc:1467] Processing CreateTablet for tablet e27ed649489145718ab95a487a880c22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=df69ffb38a98489580e5734d79c69b05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:08:46.875170 20227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e27ed649489145718ab95a487a880c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.875170 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e27ed649489145718ab95a487a880c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.875272 20431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e27ed649489145718ab95a487a880c22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:46.876565 20587 tablet_bootstrap.cc:492] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: Bootstrap starting.
I20260812 01:08:46.876951 20590 tablet_bootstrap.cc:492] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: Bootstrap starting.
I20260812 01:08:46.876953 20602 tablet_bootstrap.cc:492] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: Bootstrap starting.
I20260812 01:08:46.877276 20587 tablet_bootstrap.cc:654] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.877441 20602 tablet_bootstrap.cc:654] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.877502 20590 tablet_bootstrap.cc:654] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:46.878033 20587 tablet_bootstrap.cc:492] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: No bootstrap required, opened a new log
I20260812 01:08:46.878088 20587 ts_tablet_manager.cc:1397] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:08:46.878268 20590 tablet_bootstrap.cc:492] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: No bootstrap required, opened a new log
I20260812 01:08:46.878314 20590 ts_tablet_manager.cc:1397] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:46.878283 20587 raft_consensus.cc:348] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.878368 20587 raft_consensus.cc:374] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.878391 20587 raft_consensus.cc:729] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2756137813e43b78c4b04cc26cf53f8, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.878423 20602 tablet_bootstrap.cc:492] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: No bootstrap required, opened a new log
I20260812 01:08:46.878464 20602 ts_tablet_manager.cc:1397] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:08:46.878444 20587 consensus_queue.cc:260] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.878407 20590 raft_consensus.cc:348] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.878504 20590 raft_consensus.cc:374] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.878528 20590 raft_consensus.cc:729] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a338c1ab4a474ebe874488b6eba83c88, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.878552 20587 ts_tablet_manager.cc:1428] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:46.878592 20590 consensus_queue.cc:260] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.878615 20602 raft_consensus.cc:348] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.878618 20600 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.878698 20602 raft_consensus.cc:374] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:46.878700 20590 ts_tablet_manager.cc:1428] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:46.878718 20602 raft_consensus.cc:729] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2957c9251cd4927ba0eea3bcdb0cf62, State: Initialized, Role: FOLLOWER
I20260812 01:08:46.878715 20600 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.878762 20602 consensus_queue.cc:260] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.878835 20602 ts_tablet_manager.cc:1428] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:46.878863 20610 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.878955 20610 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.878945 20605 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.879015 20605 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.927891 20605 raft_consensus.cc:482] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:46.928020 20605 raft_consensus.cc:504] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.928421 20605 leader_election.cc:288] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1:36127), a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893)
I20260812 01:08:46.932142 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.928923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:44893 (local address 127.17.130.2:49017)
0812 01:08:46.929054 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.929085 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.929291 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.929463 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.929464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.929466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.929531 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.929533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.930255 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.930256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.931149 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.931152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.931872 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.931873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.931946 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.931952 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.932007 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.932055 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260812 01:08:46.932206 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.928994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:49017 (local address 127.17.130.3:44893)
0812 01:08:46.929045 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.929048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.929300 (+ 252us) server_negotiation.cc:407] Connection header received
0812 01:08:46.929351 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.929352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.929363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.929426 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.929634 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.930165 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.931318 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.931751 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.931809 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.931818 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.932084 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.932128 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:08:46.932508 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.928855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:37431 (local address 127.17.130.1:36127)
0812 01:08:46.928950 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.928952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.928965 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:08:46.929111 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.929113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.929146 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.929267 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:46.929468 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.930023 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.931481 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.931934 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.932017 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.932024 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.932381 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.932440 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 01:08:46.932654 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27ed649489145718ab95a487a880c22" candidate_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2756137813e43b78c4b04cc26cf53f8" is_pre_election: true
I20260812 01:08:46.932758 20647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.928726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36127 (local address 127.17.130.2:37431)
0812 01:08:46.928864 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.928904 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.929011 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.929288 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.929289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.929292 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:46.929385 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.929388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.930111 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.930112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.931362 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.931365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.932087 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.932089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.932164 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.932217 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.932606 (+ 389us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.932667 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":54,"thread_start_us":84,"threads_started":1}
I20260812 01:08:46.932775 20476 raft_consensus.cc:2454] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2957c9251cd4927ba0eea3bcdb0cf62 in term 0.
I20260812 01:08:46.933023 20155 leader_election.cc:302] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2756137813e43b78c4b04cc26cf53f8, e2957c9251cd4927ba0eea3bcdb0cf62; no voters:
I20260812 01:08:46.933230 20605 raft_consensus.cc:2781] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:46.933274 20605 raft_consensus.cc:482] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:46.933298 20605 raft_consensus.cc:3037] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.933378 20032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27ed649489145718ab95a487a880c22" candidate_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a338c1ab4a474ebe874488b6eba83c88" is_pre_election: true
I20260812 01:08:46.933533 20032 raft_consensus.cc:2454] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2957c9251cd4927ba0eea3bcdb0cf62 in term 0.
I20260812 01:08:46.934172 20605 raft_consensus.cc:504] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.934321 20605 leader_election.cc:288] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [CANDIDATE]: Term 1 election: Requested vote from peers a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1:36127), a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893)
I20260812 01:08:46.934474 20032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27ed649489145718ab95a487a880c22" candidate_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a338c1ab4a474ebe874488b6eba83c88"
I20260812 01:08:46.934538 20032 raft_consensus.cc:3037] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.934543 20476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27ed649489145718ab95a487a880c22" candidate_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2756137813e43b78c4b04cc26cf53f8"
I20260812 01:08:46.934667 20476 raft_consensus.cc:3037] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:46.935274 20476 raft_consensus.cc:2454] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2957c9251cd4927ba0eea3bcdb0cf62 in term 1.
I20260812 01:08:46.935408 20032 raft_consensus.cc:2454] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2957c9251cd4927ba0eea3bcdb0cf62 in term 1.
I20260812 01:08:46.935623 20155 leader_election.cc:302] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a338c1ab4a474ebe874488b6eba83c88, e2957c9251cd4927ba0eea3bcdb0cf62; no voters:
I20260812 01:08:46.935742 20605 raft_consensus.cc:2781] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:46.935886 20605 raft_consensus.cc:686] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 LEADER]: Becoming Leader. State: Replica: e2957c9251cd4927ba0eea3bcdb0cf62, State: Running, Role: LEADER
I20260812 01:08:46.935956 20605 consensus_queue.cc:237] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } }
I20260812 01:08:46.936370 20650 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: TxnStatusTablet state changed. Reason: New leader e2957c9251cd4927ba0eea3bcdb0cf62. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.936470 20650 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:08:46.936692 20654 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 01:08:46.936836 20654 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 01:08:46.936847 19764 catalog_manager.cc:5495] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } health_report { overall_health: UNKNOWN } } }
I20260812 01:08:46.944806 20642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.941408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40487 (local address 127.0.0.1:34370)
0812 01:08:46.941509 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.941522 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.941576 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.941947 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.941948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.941950 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:46.942010 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.942013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.943057 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.943058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.943639 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.943641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.944172 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.944173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.944230 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.944247 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.944614 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.944627 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.944687 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.944740 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 01:08:46.944818 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.941676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.17.130.2:40487)
0812 01:08:46.941763 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.941766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.941778 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:08:46.941818 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.941819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.941839 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.941928 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:46.942093 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.942952 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.943730 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.944066 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.944159 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.944356 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.944501 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.944596 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.944703 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.944730 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:08:46.946623 20476 raft_consensus.cc:1261] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Refusing update from remote peer e2957c9251cd4927ba0eea3bcdb0cf62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:46.946710 20032 raft_consensus.cc:1261] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Refusing update from remote peer e2957c9251cd4927ba0eea3bcdb0cf62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:46.946964 20650 consensus_queue.cc:1035] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:46.947260 20605 consensus_queue.cc:1035] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:46.948086 20610 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: TxnStatusTablet state changed. Reason: New leader e2957c9251cd4927ba0eea3bcdb0cf62. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.948195 20610 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.948319 20600 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: TxnStatusTablet state changed. Reason: New leader e2957c9251cd4927ba0eea3bcdb0cf62. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.948417 20600 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.949409 20600 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.949573 20600 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P a2756137813e43b78c4b04cc26cf53f8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.950017 20610 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.950117 20610 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P a338c1ab4a474ebe874488b6eba83c88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:46.950075 20660 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.950181 20660 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:08:46.950240 20660 tablet_replica.cc:439] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a338c1ab4a474ebe874488b6eba83c88" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36127 } } peers { permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 } } peers { permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 } } }
I20260812 01:08:46.950284 20660 tablet_replica.cc:442] T e27ed649489145718ab95a487a880c22 P e2957c9251cd4927ba0eea3bcdb0cf62: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:08:46.950555 20614 mvcc.cc:205] Tried to move back new op lower bound from 7317491412770852864 to 7317491412730355712. Current Snapshot: MvccSnapshot[applied={T|T < 7317491412770852864 or (T in {7317491412770852864})}]
I20260812 01:08:46.958074 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.954925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40487 (local address 127.17.130.1:55157)
0812 01:08:46.955112 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.955122 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.955177 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.955317 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.955318 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.955319 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:46.955367 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.955368 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.956397 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.956398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.957001 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.957003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.957631 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.957632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.957682 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.957699 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.957903 (+ 204us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.957905 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.957950 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.957994 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260812 01:08:46.958076 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.955012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:55157 (local address 127.17.130.2:40487)
0812 01:08:46.955064 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.955066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.955188 (+ 122us) server_negotiation.cc:407] Connection header received
0812 01:08:46.955220 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.955221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.955232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.955299 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:46.955420 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.956292 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.957177 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.957540 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.957620 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.957800 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.957837 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.957897 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.957967 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.957990 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:08:46.961077 20476 raft_consensus.cc:1261] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Refusing update from remote peer a338c1ab4a474ebe874488b6eba83c88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:46.961117 20259 raft_consensus.cc:1261] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Refusing update from remote peer a338c1ab4a474ebe874488b6eba83c88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:46.961333 20610 consensus_queue.cc:1035] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2756137813e43b78c4b04cc26cf53f8" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 44893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:46.961440 20604 consensus_queue.cc:1035] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2957c9251cd4927ba0eea3bcdb0cf62" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:46.977619 20671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.973858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36127 (local address 127.17.130.2:35713)
0812 01:08:46.974081 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:46.974093 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:46.974196 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:46.974490 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:46.974491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:46.974493 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:46.974610 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.974612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.975721 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.975722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.976417 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:46.976419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.977048 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.977128 (+ 80us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:46.977211 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.977235 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.977450 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.977451 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:08:46.977487 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 01:08:46.977528 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":141,"threads_started":1}
I20260812 01:08:46.977612 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.973929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:35713 (local address 127.17.130.1:36127)
0812 01:08:46.973984 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.973987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.974169 (+ 182us) server_negotiation.cc:407] Connection header received
0812 01:08:46.974257 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.974258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.974271 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.974344 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:46.974737 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.975598 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.976542 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.976905 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.976989 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:46.977332 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:46.977379 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:46.977444 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:46.977491 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 01:08:46.977521 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:08:46.988549 19763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:46.988667 19763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:46.989415 19763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84fc952f323541148dd90363500ab3a5: Sending DeleteTablet for 3 replicas of tablet 197b4dccebf54573b340609b76d35d07
I20260812 01:08:46.989758 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet 197b4dccebf54573b340609b76d35d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:46 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:58962
I20260812 01:08:46.989773 20227 tablet_service.cc:1514] Processing DeleteTablet for tablet 197b4dccebf54573b340609b76d35d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:46 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:34368
I20260812 01:08:46.989787 20431 tablet_service.cc:1514] Processing DeleteTablet for tablet 197b4dccebf54573b340609b76d35d07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:46 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:41082
I20260812 01:08:46.989871 20626 tablet_replica.cc:330] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: stopping tablet replica
I20260812 01:08:46.989871 20628 tablet_replica.cc:330] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: stopping tablet replica
I20260812 01:08:46.989880 20627 tablet_replica.cc:330] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: stopping tablet replica
I20260812 01:08:46.989926 20627 raft_consensus.cc:2229] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:46.989925 20628 raft_consensus.cc:2229] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:46.989939 20626 raft_consensus.cc:2229] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:08:46.989964 20627 raft_consensus.cc:2258] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:46.989969 20628 raft_consensus.cc:2258] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:46.990012 20626 raft_consensus.cc:2258] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:46.990543 20627 ts_tablet_manager.cc:1905] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:46.990581 20628 ts_tablet_manager.cc:1905] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:46.992383 20628 ts_tablet_manager.cc:1918] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:46.992452 20628 log.cc:1196] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/197b4dccebf54573b340609b76d35d07
I20260812 01:08:46.992468 20626 ts_tablet_manager.cc:1905] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:46.992703 20628 ts_tablet_manager.cc:1939] T 197b4dccebf54573b340609b76d35d07 P e2957c9251cd4927ba0eea3bcdb0cf62: Deleting consensus metadata
I20260812 01:08:46.993044 19747 catalog_manager.cc:4841] TS e2957c9251cd4927ba0eea3bcdb0cf62 (127.17.130.2:40487): tablet 197b4dccebf54573b340609b76d35d07 (table test-table [id=ea88f4a65be44ce7913a20efde903d15]) successfully deleted
I20260812 01:08:46.993269 20627 ts_tablet_manager.cc:1918] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:46.993331 20627 log.cc:1196] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/197b4dccebf54573b340609b76d35d07
I20260812 01:08:46.993577 20627 ts_tablet_manager.cc:1939] T 197b4dccebf54573b340609b76d35d07 P a2756137813e43b78c4b04cc26cf53f8: Deleting consensus metadata
I20260812 01:08:46.993933 19748 catalog_manager.cc:4841] TS a2756137813e43b78c4b04cc26cf53f8 (127.17.130.3:44893): tablet 197b4dccebf54573b340609b76d35d07 (table test-table [id=ea88f4a65be44ce7913a20efde903d15]) successfully deleted
I20260812 01:08:46.993974 20626 ts_tablet_manager.cc:1918] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:46.994026 20626 log.cc:1196] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/197b4dccebf54573b340609b76d35d07
I20260812 01:08:46.994263 20626 ts_tablet_manager.cc:1939] T 197b4dccebf54573b340609b76d35d07 P a338c1ab4a474ebe874488b6eba83c88: Deleting consensus metadata
I20260812 01:08:46.994602 19748 catalog_manager.cc:4841] TS a338c1ab4a474ebe874488b6eba83c88 (127.17.130.1:36127): tablet 197b4dccebf54573b340609b76d35d07 (table test-table [id=ea88f4a65be44ce7913a20efde903d15]) successfully deleted
Aug 12 01:08:46 dist-test-slave-tfpf krb5kdc[19695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496926, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:08:47.003710 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:46.994266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58980 (local address 127.17.130.1:36127)
0812 01:08:46.994316 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:46.994318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:46.994709 (+ 391us) server_negotiation.cc:407] Connection header received
0812 01:08:46.994759 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:46.994760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:46.994774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:46.994832 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:46.995419 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.995915 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.996728 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:46.996936 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:46.997025 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:47.000259 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:47.000274 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:47.000274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:47.000284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:47.002194 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:47.002631 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:47.002636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:47.002636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:47.002755 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:47.002981 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:47.002982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:47.002983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:47.003236 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:47.003413 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:47.003482 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 01:08:47.003589 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260812 01:08:47.004045 20095 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58980
I20260812 01:08:47.012110 19853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:47.005306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32810 (local address 127.17.130.62:41877)
0812 01:08:47.005359 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:47.005361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:47.005425 (+ 64us) server_negotiation.cc:407] Connection header received
0812 01:08:47.005505 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:47.005506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:47.005517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:47.005573 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:47.006106 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:47.006624 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:47.007330 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:47.007547 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:47.007644 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:47.009032 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:47.009033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:47.009034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:47.009042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:47.010594 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:47.011170 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:47.011171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:47.011171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:47.011244 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:47.011475 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:47.011476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:47.011476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:47.011604 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:47.011716 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:47.011881 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 01:08:47.011977 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:08:47.012501 19763 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32810
I20260812 01:08:47.013239 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19895
I20260812 01:08:47.034752 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20122
I20260812 01:08:47.050818 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20341
I20260812 01:08:47.068323 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19711
2026-08-12T01:08:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2819 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:47 dist-test-slave-tfpf krb5kdc[20689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:47 dist-test-slave-tfpf krb5kdc[20689](info): set up 2 sockets
Aug 12 01:08:47 dist-test-slave-tfpf krb5kdc[20689](info): commencing operation
krb5kdc: starting...
W20260812 01:08:48.409754 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:08:48 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:08:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:08:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:08:48.562259 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44321
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:48.670010 20706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:48.670233 20706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:48.670257 20706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:48.671398 20706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:08:48.671437 20706 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 01:08:48.671450 20706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:48.671463 20706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:48.671476 20706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:08:48.671489 20706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:08:48.672991 20706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44321
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:48.673307 20706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:48.673530 20706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:48.679862 20719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:48.679865 20715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:48.679865 20712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:48.680420 20706 server_base.cc:1034] running on GCE node
I20260812 01:08:48.680763 20706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:48.681124 20706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:48.682313 20706 hybrid_clock.cc:648] HybridClock initialized: now 1786496928682265 us; error 64 us; skew 500 ppm
Aug 12 01:08:48 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496928, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:48.690347 20706 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:08:48.690910 20706 webserver.cc:460] Webserver started at http://127.17.130.62:46413/ using document root <none> and password file <none>
I20260812 01:08:48.691291 20706 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:48.691365 20706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:48.691526 20706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:48.692873 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "331ee5ce8d464b06869522429586d78f"
format_stamp: "Formatted at 2026-08-12 01:08:48 on dist-test-slave-tfpf"
server_key: "8f98ef109514b6a2557655750c5c4486"
server_key_iv: "ba0d0e356954a54854c71da5d2a5fad2"
server_key_version: "encryptionkey@0"
I20260812 01:08:48.693395 20706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "331ee5ce8d464b06869522429586d78f"
format_stamp: "Formatted at 2026-08-12 01:08:48 on dist-test-slave-tfpf"
server_key: "8f98ef109514b6a2557655750c5c4486"
server_key_iv: "ba0d0e356954a54854c71da5d2a5fad2"
server_key_version: "encryptionkey@0"
I20260812 01:08:48.695274 20706 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 01:08:48.696293 20729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:48.696547 20706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:48.696691 20706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "331ee5ce8d464b06869522429586d78f"
format_stamp: "Formatted at 2026-08-12 01:08:48 on dist-test-slave-tfpf"
server_key: "8f98ef109514b6a2557655750c5c4486"
server_key_iv: "ba0d0e356954a54854c71da5d2a5fad2"
server_key_version: "encryptionkey@0"
I20260812 01:08:48.696785 20706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:48.708041 20706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:48.710688 20706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:48.710878 20706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:48.716578 20706 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44321
I20260812 01:08:48.716667 20832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44321 every 8 connection(s)
I20260812 01:08:48.717161 20706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:08:48.718039 20835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:48.719971 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20706
I20260812 01:08:48.720078 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:08:48.720249 17928 external_mini_cluster.cc:1428] Setting key a5b2c53abf3e9c887f5c7f5f26766eac
I20260812 01:08:48.721174 20835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Bootstrap starting.
I20260812 01:08:48.721928 20835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:48.722293 20835 log.cc:824] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:48.723204 20835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: No bootstrap required, opened a new log
I20260812 01:08:48.725693 20835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "331ee5ce8d464b06869522429586d78f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44321 } }
I20260812 01:08:48.725929 20835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:48.725975 20835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 331ee5ce8d464b06869522429586d78f, State: Initialized, Role: FOLLOWER
I20260812 01:08:48.726154 20835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "331ee5ce8d464b06869522429586d78f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44321 } }
I20260812 01:08:48.726269 20835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:08:48.726320 20835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:08:48.726379 20835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:48.727299 20835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "331ee5ce8d464b06869522429586d78f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44321 } }
I20260812 01:08:48.727435 20835 leader_election.cc:302] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 331ee5ce8d464b06869522429586d78f; no voters:
I20260812 01:08:48.727659 20835 leader_election.cc:288] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:08:48.727725 20842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:48.727895 20842 raft_consensus.cc:686] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [term 1 LEADER]: Becoming Leader. State: Replica: 331ee5ce8d464b06869522429586d78f, State: Running, Role: LEADER
I20260812 01:08:48.728003 20835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:08:48.728017 20842 consensus_queue.cc:237] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "331ee5ce8d464b06869522429586d78f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44321 } }
Aug 12 01:08:48 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:48.728492 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 331ee5ce8d464b06869522429586d78f. Latest consensus state: current_term: 1 leader_uuid: "331ee5ce8d464b06869522429586d78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "331ee5ce8d464b06869522429586d78f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44321 } } }
I20260812 01:08:48.728578 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:48.729249 20850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:08:48.729437 20843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "331ee5ce8d464b06869522429586d78f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "331ee5ce8d464b06869522429586d78f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44321 } } }
I20260812 01:08:48.729592 20843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:48.730181 20850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:08:48.732100 20850 catalog_manager.cc:1347] Generated new cluster ID: 7ea0777d85ed49d1ae98affe9e43b403
I20260812 01:08:48.732179 20850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:08:48.733101 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:48.720830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37648 (local address 127.17.130.62:44321)
0812 01:08:48.721038 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:48.721047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:48.721106 (+ 59us) server_negotiation.cc:407] Connection header received
0812 01:08:48.721629 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:48.721636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:48.721689 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:48.721811 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:48.722671 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.723873 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.725154 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.725605 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.725701 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:48.728981 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:48.728986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:48.728986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:48.729000 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:48.731171 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:48.731681 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:48.731684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:48.731685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:48.731798 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:48.732043 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:48.732044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:48.732045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:48.732373 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:48.732511 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:48.732704 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 01:08:48.732864 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260812 01:08:48.744594 20850 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:08:48.745270 20850 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:08:48.751364 20850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Generated new TSK 0
I20260812 01:08:48.751617 20850 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:08:48.786643 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44321
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:48.905752 20873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:48.905982 20873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:48.906011 20873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:48.907238 20873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:48.907289 20873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:48.907312 20873 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 01:08:48.907325 20873 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:08:48.907343 20873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:08:48.908963 20873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:44321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:48.909258 20873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:48.909484 20873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:48.915388 20880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:48.915516 20873 server_base.cc:1034] running on GCE node
W20260812 01:08:48.915388 20882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:48.915520 20879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:48.915953 20873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:48.916225 20873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:48.917411 20873 hybrid_clock.cc:648] HybridClock initialized: now 1786496928917393 us; error 29 us; skew 500 ppm
Aug 12 01:08:48 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496928, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:48.920385 20873 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:08:48.920845 20873 webserver.cc:460] Webserver started at http://127.17.130.1:41747/ using document root <none> and password file <none>
I20260812 01:08:48.921180 20873 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:48.921255 20873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:48.921422 20873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:48.922580 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "eba2620bdf2d46119e2181ccdff1c10f"
format_stamp: "Formatted at 2026-08-12 01:08:48 on dist-test-slave-tfpf"
server_key: "9c54c428300cd5d338155871f018b323"
server_key_iv: "b3523f6805c3b158784901b40f5fce7b"
server_key_version: "encryptionkey@0"
I20260812 01:08:48.922978 20873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "eba2620bdf2d46119e2181ccdff1c10f"
format_stamp: "Formatted at 2026-08-12 01:08:48 on dist-test-slave-tfpf"
server_key: "9c54c428300cd5d338155871f018b323"
server_key_iv: "b3523f6805c3b158784901b40f5fce7b"
server_key_version: "encryptionkey@0"
I20260812 01:08:48.924539 20873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:08:48.925551 20892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:48.925843 20873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:08:48.925987 20873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "eba2620bdf2d46119e2181ccdff1c10f"
format_stamp: "Formatted at 2026-08-12 01:08:48 on dist-test-slave-tfpf"
server_key: "9c54c428300cd5d338155871f018b323"
server_key_iv: "b3523f6805c3b158784901b40f5fce7b"
server_key_version: "encryptionkey@0"
I20260812 01:08:48.926108 20873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:48.940783 20873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:48.943567 20873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:48.943771 20873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:48.944397 20873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:48.944442 20873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:48.944468 20873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:48.944480 20873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:48.952111 20873 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:39701
I20260812 01:08:48.952265 21067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:39701 every 8 connection(s)
Aug 12 01:08:48 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496928, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:48.952615 20873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:08:48.954165 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20873
I20260812 01:08:48.954277 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:08:48.954483 17928 external_mini_cluster.cc:1428] Setting key b67eee021a26fff9123f725bda329909
I20260812 01:08:48.956898 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:48.944936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.17.130.1:40323)
0812 01:08:48.945204 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:48.945248 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:48.945761 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:48.946075 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:48.946078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:48.946504 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:48.946835 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:48.946844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.948031 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.948033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:48.949544 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:48.949549 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.950150 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.950151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:48.950282 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:48.951057 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:48.951070 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:48.953101 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:48.955233 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:48.955243 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:48.955245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:48.955672 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:48.955927 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:48.955928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:48.955928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:48.956037 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:48.956351 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:48.956356 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:48.956486 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 01:08:48.956570 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 01:08:48.956722 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":145,"threads_started":1}
I20260812 01:08:48.956905 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:48.945017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:40323 (local address 127.17.130.62:44321)
0812 01:08:48.945110 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:48.945113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:48.945350 (+ 237us) server_negotiation.cc:407] Connection header received
0812 01:08:48.945878 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:48.945879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:48.945895 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:48.945977 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:48.946967 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.947876 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.949673 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.950013 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.950117 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:48.953206 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:48.953208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:48.953209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:48.953218 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:48.954833 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:48.955798 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:48.955799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:48.955799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:48.955868 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:48.956120 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:48.956121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:48.956122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:48.956285 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:48.956353 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:48.956632 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 01:08:48.956760 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:08:48.957516 21068 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44321
I20260812 01:08:48.957618 21068 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:48.957839 21068 heartbeater.cc:511] Master 127.17.130.62:44321 requested a full tablet report, sending...
I20260812 01:08:48.959115 20751 ts_manager.cc:194] Registered new tserver with Master: eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701)
I20260812 01:08:48.960180 20751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:40323
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
I20260812 01:08:48.967497 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:48.959805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:53453 (local address 127.17.130.62:44321)
0812 01:08:48.959846 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:48.959848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:48.960003 (+ 155us) server_negotiation.cc:407] Connection header received
0812 01:08:48.960056 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:48.960057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:48.960069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:48.960129 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:48.960583 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.961449 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.962229 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.962567 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.962651 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:48.963761 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:48.963763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:48.963764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:48.963773 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:48.965570 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:48.966324 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:48.966326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:48.966327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:48.966442 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:48.966698 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:48.966699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:48.966700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:48.966863 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:48.966963 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:48.967230 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 01:08:48.967372 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:08:48.967509 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:48.959738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.17.130.1:53453)
0812 01:08:48.959914 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:48.959926 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:48.959999 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:48.960150 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:48.960151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:48.960377 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:48.960460 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:48.960463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.961574 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.961576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:48.962135 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:48.962137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:48.962680 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:48.962681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:48.962749 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:48.963307 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:48.963312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:48.963663 (+ 351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:48.965670 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:48.965671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:48.965671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:48.966105 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:48.966541 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:48.966542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:48.966543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:48.966616 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:48.966961 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:48.966962 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:48.967056 (+ 94us) client_negotiation.cc:769] Sending connection context
0812 01:08:48.967180 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 01:08:48.967376 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:08:48.992730 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44321
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:49.087538 21084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:49.087767 21084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:49.087795 21084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:49.089260 21084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:49.089329 21084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:49.089354 21084 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 01:08:49.089365 21084 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:08:49.089385 21084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:08:49.091161 21084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:44321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:49.091444 21084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:49.091714 21084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:49.097398 21094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:49.097398 21096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:49.097398 21093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:49.097622 21084 server_base.cc:1034] running on GCE node
I20260812 01:08:49.097920 21084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:49.098196 21084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:49.099368 21084 hybrid_clock.cc:648] HybridClock initialized: now 1786496929099356 us; error 30 us; skew 500 ppm
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:49.102517 21084 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:08:49.103149 21084 webserver.cc:460] Webserver started at http://127.17.130.2:34911/ using document root <none> and password file <none>
I20260812 01:08:49.103430 21084 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:49.103507 21084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:49.103667 21084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:49.104784 21084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "0392bee06ee742758c136184a537c205"
format_stamp: "Formatted at 2026-08-12 01:08:49 on dist-test-slave-tfpf"
server_key: "fc6ccfe5bc614f5e69495afa35e34be8"
server_key_iv: "4413f351cede489a2175a3ad3cfbb944"
server_key_version: "encryptionkey@0"
I20260812 01:08:49.105249 21084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "0392bee06ee742758c136184a537c205"
format_stamp: "Formatted at 2026-08-12 01:08:49 on dist-test-slave-tfpf"
server_key: "fc6ccfe5bc614f5e69495afa35e34be8"
server_key_iv: "4413f351cede489a2175a3ad3cfbb944"
server_key_version: "encryptionkey@0"
I20260812 01:08:49.106812 21084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:49.107762 21105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:49.108152 21084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:49.108305 21084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "0392bee06ee742758c136184a537c205"
format_stamp: "Formatted at 2026-08-12 01:08:49 on dist-test-slave-tfpf"
server_key: "fc6ccfe5bc614f5e69495afa35e34be8"
server_key_iv: "4413f351cede489a2175a3ad3cfbb944"
server_key_version: "encryptionkey@0"
I20260812 01:08:49.108425 21084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:49.119992 21084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:49.122587 21084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:49.122786 21084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:49.123462 21084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:49.123535 21084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:49.123581 21084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:49.123606 21084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:49.131701 21084 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:40099
I20260812 01:08:49.131798 21294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:40099 every 8 connection(s)
I20260812 01:08:49.132117 21084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:08:49.134747 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.123933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:51967 (local address 127.17.130.62:44321)
0812 01:08:49.124024 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.124027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.124243 (+ 216us) server_negotiation.cc:407] Connection header received
0812 01:08:49.124738 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.124739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.124753 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.124846 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:49.125625 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.126496 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.127673 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.127877 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.127979 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.131509 (+ 3530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:49.131510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:49.131511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:49.131520 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:49.133286 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.133721 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.133722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.133722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.133787 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.134050 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.134051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.134051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.134190 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:49.134273 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.134559 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.134645 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:08:49.134951 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.123912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.17.130.2:51967)
0812 01:08:49.124089 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.124132 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.124629 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.124899 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.124903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.125256 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:49.125515 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.125521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.126635 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.126636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.127575 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.127579 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.127973 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.127973 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.128140 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.128796 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:49.128809 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:49.131390 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:49.133375 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.133383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.133385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.133623 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.133863 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.133864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.133864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.133982 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.134263 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:49.134270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:49.134416 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.134525 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.134713 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260812 01:08:49.135582 21296 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44321
I20260812 01:08:49.135679 21296 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:49.135882 21296 heartbeater.cc:511] Master 127.17.130.62:44321 requested a full tablet report, sending...
I20260812 01:08:49.136309 20751 ts_manager.cc:194] Registered new tserver with Master: 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:49.136724 20751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:51967
I20260812 01:08:49.141609 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21084
I20260812 01:08:49.141711 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:08:49.141927 17928 external_mini_cluster.cc:1428] Setting key d646e5cf964b6574436370d01fc961c2
I20260812 01:08:49.143748 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.136869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:39409 (local address 127.17.130.62:44321)
0812 01:08:49.136910 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.136912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.137083 (+ 171us) server_negotiation.cc:407] Connection header received
0812 01:08:49.137111 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.137112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.137123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.137189 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:49.137563 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.138038 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.138739 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.139005 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.139080 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.140114 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:49.140115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:49.140115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:49.140122 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:49.142401 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.142828 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.142830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.142830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.142895 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.143136 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.143137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.143138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.143305 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:49.143388 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.143561 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.143651 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:08:49.143751 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.136817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.17.130.2:39409)
0812 01:08:49.136965 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.136977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.137036 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.137199 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.137200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.137402 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:49.137477 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.137479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.138143 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.138144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.138642 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.138643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.139089 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.139090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.139177 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.139683 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:49.139689 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:49.139999 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:49.142504 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.142505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.142506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.142751 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.142983 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.142984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.142985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.143066 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.143378 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:49.143379 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:49.143431 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.143523 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.143646 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:08:49.172879 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44321
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:49.267580 21312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:49.267835 21312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:49.267884 21312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:49.269163 21312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:49.269232 21312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:49.269270 21312 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 01:08:49.269307 21312 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:08:49.269341 21312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:08:49.270841 21312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:44321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:49.271055 21312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:49.271251 21312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:49.278116 21321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:49.278146 21327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:49.278146 21322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:49.279217 21312 server_base.cc:1034] running on GCE node
I20260812 01:08:49.279502 21312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:49.279786 21312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:49.280987 21312 hybrid_clock.cc:648] HybridClock initialized: now 1786496929280968 us; error 47 us; skew 500 ppm
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:49.283733 21312 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:08:49.284286 21312 webserver.cc:460] Webserver started at http://127.17.130.3:36273/ using document root <none> and password file <none>
I20260812 01:08:49.284529 21312 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:49.284602 21312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:49.284781 21312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:49.286018 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "c856d245d4a14a31b980a38700296a09"
format_stamp: "Formatted at 2026-08-12 01:08:49 on dist-test-slave-tfpf"
server_key: "f5b6c5ab985758c425c6f17a50f2eb21"
server_key_iv: "cfb5e1e3cdbb5fbb9794cdb13f5990f0"
server_key_version: "encryptionkey@0"
I20260812 01:08:49.286422 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "c856d245d4a14a31b980a38700296a09"
format_stamp: "Formatted at 2026-08-12 01:08:49 on dist-test-slave-tfpf"
server_key: "f5b6c5ab985758c425c6f17a50f2eb21"
server_key_iv: "cfb5e1e3cdbb5fbb9794cdb13f5990f0"
server_key_version: "encryptionkey@0"
I20260812 01:08:49.287976 21312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:08:49.288934 21344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:49.289228 21312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:49.289327 21312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "c856d245d4a14a31b980a38700296a09"
format_stamp: "Formatted at 2026-08-12 01:08:49 on dist-test-slave-tfpf"
server_key: "f5b6c5ab985758c425c6f17a50f2eb21"
server_key_iv: "cfb5e1e3cdbb5fbb9794cdb13f5990f0"
server_key_version: "encryptionkey@0"
I20260812 01:08:49.289389 21312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:49.300792 21312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:49.303499 21312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:49.303709 21312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:49.304294 21312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:49.304363 21312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:49.304428 21312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:49.304461 21312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496929, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:49.312557 21312 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:36331
I20260812 01:08:49.312702 21532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:36331 every 8 connection(s)
I20260812 01:08:49.313017 21312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:08:49.316457 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.304862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:58895 (local address 127.17.130.62:44321)
0812 01:08:49.304937 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.304940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.305217 (+ 277us) server_negotiation.cc:407] Connection header received
0812 01:08:49.305755 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.305756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.305768 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.305826 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:49.306695 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.307501 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.309096 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.309318 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.309438 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.312355 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:49.312356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:49.312357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:49.312364 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:49.314700 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.315366 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.315367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.315367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.315436 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.315777 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.315778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.315778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.315904 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:49.315965 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.316317 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.316387 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:08:49.316712 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.304854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.17.130.3:58895)
0812 01:08:49.305049 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.305118 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.305643 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.305903 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.305907 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.306302 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:49.306608 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.306614 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.307596 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.307598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.308976 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.308979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.309514 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.309516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.309704 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.310297 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:49.310307 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:49.311971 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:49.314834 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.314842 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.314844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.315264 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.315564 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.315565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.315565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.315705 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.316008 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:49.316014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:49.316165 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.316268 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.316406 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260812 01:08:49.317549 21533 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44321
I20260812 01:08:49.317675 21533 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:49.317888 21533 heartbeater.cc:511] Master 127.17.130.62:44321 requested a full tablet report, sending...
I20260812 01:08:49.318588 20751 ts_manager.cc:194] Registered new tserver with Master: c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331)
I20260812 01:08:49.319123 20751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:58895
I20260812 01:08:49.320142 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21312
I20260812 01:08:49.320218 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:08:49.320428 17928 external_mini_cluster.cc:1428] Setting key df9cef81b27d72ee0fecdb507ad8c10b
I20260812 01:08:49.321283 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:08:49.325984 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.319125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:47175 (local address 127.17.130.62:44321)
0812 01:08:49.319166 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.319168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.319200 (+ 32us) server_negotiation.cc:407] Connection header received
0812 01:08:49.319319 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.319320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.319330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.319376 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:49.319784 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.320345 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.321434 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.321647 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.321701 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.323115 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:49.323116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:49.323117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:49.323124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:49.324645 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.325053 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.325054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.325055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.325154 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.325381 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.325382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.325382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.325512 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:49.325592 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.325789 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.325876 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:08:49.326009 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.318988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.17.130.3:47175)
0812 01:08:49.319140 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.319152 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.319215 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.319392 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.319394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.319627 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:49.319702 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.319705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.320443 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.320444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.321352 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.321356 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.321728 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.321735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.321833 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.322474 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:49.322481 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:49.322997 (+ 516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:49.324753 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.324755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.324755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.324956 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.325234 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.325235 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.325235 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.325309 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.325584 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:49.325585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:49.325655 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.325740 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.325879 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:49.351531 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.342868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.17.130.62:44321)
0812 01:08:49.342968 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.342970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.343101 (+ 131us) server_negotiation.cc:407] Connection header received
0812 01:08:49.343199 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.343200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.343212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.343283 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:49.344010 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.344543 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.345459 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.345661 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.345732 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.348365 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:49.348367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:49.348367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:49.348374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:49.349923 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.350292 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.350293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.350293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.350346 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.350595 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.350596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.350596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.350716 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:49.350799 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.351115 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.351419 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 01:08:49.353477 20751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:08:49.355068 20751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:49.362919 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.358242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.17.130.2:40099)
0812 01:08:49.358591 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.358597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.358608 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:08:49.358659 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.358661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.358679 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.358772 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.359044 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.360227 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.361902 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.362417 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.362514 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.362583 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.362750 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.362795 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":89,"threads_started":1}
I20260812 01:08:49.362907 21562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.358092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.0.0.1:45462)
0812 01:08:49.358420 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.358430 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.358493 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.358794 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.358797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.358799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.358954 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.358956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.360327 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.360328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.361839 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.361841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.362520 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.362521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.362608 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.362624 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.362730 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.362795 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":45,"thread_start_us":207,"threads_started":1}
I20260812 01:08:49.363001 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.358314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.17.130.3:36331)
0812 01:08:49.359137 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.359141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.359151 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:08:49.359192 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.359194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.359211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.359291 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.359448 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.360492 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.361823 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.362534 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.362596 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.362649 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.362859 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.362909 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":111,"threads_started":1}
I20260812 01:08:49.363113 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.358646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33306 (local address 127.17.130.1:39701)
0812 01:08:49.358839 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.358843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.358887 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:08:49.359393 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.359396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.359414 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.359484 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.359592 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.360334 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.361662 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.362440 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.362515 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.362583 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.362948 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.363001 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260812 01:08:49.362911 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.358428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.0.0.1:33306)
0812 01:08:49.358831 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.358841 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.358897 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.359490 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.359490 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.359492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.359539 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.359541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.360424 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.360425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.361570 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.361573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.362622 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.362623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.362706 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.362710 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.362762 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.362811 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":42,"thread_start_us":58,"threads_started":1}
I20260812 01:08:49.363170 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.358061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36331 (local address 127.0.0.1:51604)
0812 01:08:49.358373 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.358418 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.358493 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.359303 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.359303 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.359305 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.359362 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.359364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.360575 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.360576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.361733 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.361735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.362678 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.362679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.362733 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.362736 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.362782 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.362812 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":4,"thread_start_us":87,"threads_started":1}
I20260812 01:08:49.364022 21424 tablet_service.cc:1467] Processing CreateTablet for tablet e1c4df153d34459792f318d67c54046d (DEFAULT_TABLE table=test-table [id=f986952939c84ab4a5d4041d98a47391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:49.364415 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1c4df153d34459792f318d67c54046d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.364331 20972 tablet_service.cc:1467] Processing CreateTablet for tablet e1c4df153d34459792f318d67c54046d (DEFAULT_TABLE table=test-table [id=f986952939c84ab4a5d4041d98a47391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:49.364550 21190 tablet_service.cc:1467] Processing CreateTablet for tablet e1c4df153d34459792f318d67c54046d (DEFAULT_TABLE table=test-table [id=f986952939c84ab4a5d4041d98a47391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:49.364704 20972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1c4df153d34459792f318d67c54046d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.364894 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1c4df153d34459792f318d67c54046d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.367266 21572 tablet_bootstrap.cc:492] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Bootstrap starting.
I20260812 01:08:49.368088 21572 tablet_bootstrap.cc:654] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.368337 21572 log.cc:824] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:49.369037 21572 tablet_bootstrap.cc:492] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: No bootstrap required, opened a new log
I20260812 01:08:49.369170 21572 ts_tablet_manager.cc:1397] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:08:49.369459 21576 tablet_bootstrap.cc:492] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Bootstrap starting.
I20260812 01:08:49.369915 21577 tablet_bootstrap.cc:492] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Bootstrap starting.
I20260812 01:08:49.370309 21576 tablet_bootstrap.cc:654] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.370633 21577 tablet_bootstrap.cc:654] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.370651 21576 log.cc:824] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:49.370980 21577 log.cc:824] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:49.371619 21572 raft_consensus.cc:348] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.371798 21577 tablet_bootstrap.cc:492] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: No bootstrap required, opened a new log
I20260812 01:08:49.371822 21572 raft_consensus.cc:374] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.371861 21577 ts_tablet_manager.cc:1397] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:08:49.371862 21572 raft_consensus.cc:729] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c856d245d4a14a31b980a38700296a09, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.372162 21572 consensus_queue.cc:260] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.372507 21572 ts_tablet_manager.cc:1428] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:08:49.372596 21533 heartbeater.cc:503] Master 127.17.130.62:44321 was elected leader, sending a full tablet report...
I20260812 01:08:49.373356 21576 tablet_bootstrap.cc:492] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: No bootstrap required, opened a new log
I20260812 01:08:49.373454 21576 ts_tablet_manager.cc:1397] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:08:49.374580 21577 raft_consensus.cc:348] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.374724 21577 raft_consensus.cc:374] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.374747 21577 raft_consensus.cc:729] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba2620bdf2d46119e2181ccdff1c10f, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.374876 21577 consensus_queue.cc:260] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.375190 21577 ts_tablet_manager.cc:1428] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 01:08:49.375223 21068 heartbeater.cc:503] Master 127.17.130.62:44321 was elected leader, sending a full tablet report...
I20260812 01:08:49.375355 21576 raft_consensus.cc:348] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.375494 21576 raft_consensus.cc:374] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.375545 21576 raft_consensus.cc:729] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0392bee06ee742758c136184a537c205, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.375661 21576 consensus_queue.cc:260] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.375957 21576 ts_tablet_manager.cc:1428] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:08:49.376037 21296 heartbeater.cc:503] Master 127.17.130.62:44321 was elected leader, sending a full tablet report...
W20260812 01:08:49.383054 21298 tablet.cc:2369] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:08:49.392258 21582 raft_consensus.cc:482] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:49.392387 21582 raft_consensus.cc:504] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.392822 21582 leader_election.cc:288] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0392bee06ee742758c136184a537c205 (127.17.130.2:40099), c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331)
I20260812 01:08:49.396349 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.393246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:59569 (local address 127.17.130.2:40099)
0812 01:08:49.393312 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.393315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.393593 (+ 278us) server_negotiation.cc:407] Connection header received
0812 01:08:49.393676 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.393677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.393686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.393742 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.393903 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.394392 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.395540 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.395999 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.396078 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.396088 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.396242 (+ 154us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.396288 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:08:49.396340 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.393150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36331 (local address 127.17.130.1:41481)
0812 01:08:49.393206 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.393220 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.393273 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.393443 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.393445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.393447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.393527 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.393529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.394169 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.394171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.395354 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.395357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.396034 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.396035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.396134 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.396138 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.396200 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.396248 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 01:08:49.396353 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.393187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:41481 (local address 127.17.130.3:36331)
0812 01:08:49.393287 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.393289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.393301 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:08:49.393335 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.393336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.393361 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.393437 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.393607 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.394085 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.395458 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.395867 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.395959 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.395971 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.396220 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.396273 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:08:49.396689 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.393157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.17.130.1:59569)
0812 01:08:49.393506 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.393524 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.393598 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.393746 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.393746 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.393747 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.393831 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.393833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.394495 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.394496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.395440 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.395443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.396073 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.396073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.396149 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.396152 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.396488 (+ 336us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.396534 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":168,"thread_start_us":131,"threads_started":1}
I20260812 01:08:49.396781 21457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1c4df153d34459792f318d67c54046d" candidate_uuid: "eba2620bdf2d46119e2181ccdff1c10f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c856d245d4a14a31b980a38700296a09" is_pre_election: true
I20260812 01:08:49.397017 21457 raft_consensus.cc:2454] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eba2620bdf2d46119e2181ccdff1c10f in term 0.
I20260812 01:08:49.397101 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1c4df153d34459792f318d67c54046d" candidate_uuid: "eba2620bdf2d46119e2181ccdff1c10f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205" is_pre_election: true
I20260812 01:08:49.397241 21218 raft_consensus.cc:2454] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eba2620bdf2d46119e2181ccdff1c10f in term 0.
I20260812 01:08:49.397480 20893 leader_election.cc:302] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0392bee06ee742758c136184a537c205, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:49.397667 21582 raft_consensus.cc:2781] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:49.397737 21582 raft_consensus.cc:482] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:49.397758 21582 raft_consensus.cc:3037] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.398520 21582 raft_consensus.cc:504] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.398692 21582 leader_election.cc:288] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [CANDIDATE]: Term 1 election: Requested vote from peers 0392bee06ee742758c136184a537c205 (127.17.130.2:40099), c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331)
I20260812 01:08:49.398833 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1c4df153d34459792f318d67c54046d" candidate_uuid: "eba2620bdf2d46119e2181ccdff1c10f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205"
I20260812 01:08:49.398867 21457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1c4df153d34459792f318d67c54046d" candidate_uuid: "eba2620bdf2d46119e2181ccdff1c10f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c856d245d4a14a31b980a38700296a09"
I20260812 01:08:49.398957 21457 raft_consensus.cc:3037] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.399039 21218 raft_consensus.cc:3037] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.399820 21457 raft_consensus.cc:2454] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eba2620bdf2d46119e2181ccdff1c10f in term 1.
I20260812 01:08:49.399940 21218 raft_consensus.cc:2454] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eba2620bdf2d46119e2181ccdff1c10f in term 1.
I20260812 01:08:49.400481 20895 leader_election.cc:302] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c856d245d4a14a31b980a38700296a09, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:49.400611 21582 raft_consensus.cc:2781] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:49.400889 21582 raft_consensus.cc:686] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 1 LEADER]: Becoming Leader. State: Replica: eba2620bdf2d46119e2181ccdff1c10f, State: Running, Role: LEADER
I20260812 01:08:49.401013 21582 consensus_queue.cc:237] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } }
I20260812 01:08:49.401998 20752 catalog_manager.cc:5495] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f reported cstate change: term changed from 0 to 1, leader changed from <none> to eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1). New cstate: current_term: 1 leader_uuid: "eba2620bdf2d46119e2181ccdff1c10f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } health_report { overall_health: HEALTHY } } }
Aug 12 01:08:49 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496928, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:49.434355 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.426501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.17.130.62:44321)
0812 01:08:49.426624 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.426627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.426852 (+ 225us) server_negotiation.cc:407] Connection header received
0812 01:08:49.426966 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.426967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.426984 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.427050 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:49.427571 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.428086 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.428861 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.429046 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.429161 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.431598 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:49.431600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:49.431600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:49.431609 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:49.432952 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.433418 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.433419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.433420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.433480 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:49.433693 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:49.433694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:49.433694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:49.433815 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:49.433897 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.434070 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.434263 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":89}
I20260812 01:08:49.434356 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.426393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44321 (local address 127.0.0.1:37664)
0812 01:08:49.426658 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.426676 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.426876 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.427056 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.427057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.427395 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:49.427481 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.427484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.428172 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.428174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.428766 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.428768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.429160 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.429161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.429239 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.429986 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:49.429991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:49.431474 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:49.433102 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.433111 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.433115 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.433324 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.433549 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:49.433550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:49.433551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:49.433626 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:49.433884 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:49.433893 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:49.433943 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.434040 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.434208 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":2,"thread_start_us":155,"threads_started":1}
I20260812 01:08:49.437459 20752 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:37664:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 01:08:49.437680 20752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:49.439908 20972 tablet_service.cc:1467] Processing CreateTablet for tablet baeafcd4f8a545b4b9fccd023ab60e02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56b6b5bff13c40979ef128bbe45633af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:08:49.439960 21424 tablet_service.cc:1467] Processing CreateTablet for tablet baeafcd4f8a545b4b9fccd023ab60e02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56b6b5bff13c40979ef128bbe45633af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:08:49.440147 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baeafcd4f8a545b4b9fccd023ab60e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.440147 20972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baeafcd4f8a545b4b9fccd023ab60e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.440299 21190 tablet_service.cc:1467] Processing CreateTablet for tablet baeafcd4f8a545b4b9fccd023ab60e02 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=56b6b5bff13c40979ef128bbe45633af]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:08:49.440537 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baeafcd4f8a545b4b9fccd023ab60e02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.441859 21577 tablet_bootstrap.cc:492] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: Bootstrap starting.
I20260812 01:08:49.442406 21577 tablet_bootstrap.cc:654] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.442828 21572 tablet_bootstrap.cc:492] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: Bootstrap starting.
I20260812 01:08:49.442842 21576 tablet_bootstrap.cc:492] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: Bootstrap starting.
I20260812 01:08:49.443392 21572 tablet_bootstrap.cc:654] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.443544 21576 tablet_bootstrap.cc:654] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.443948 21577 tablet_bootstrap.cc:492] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: No bootstrap required, opened a new log
I20260812 01:08:49.444025 21577 ts_tablet_manager.cc:1397] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:08:49.444101 21572 tablet_bootstrap.cc:492] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: No bootstrap required, opened a new log
I20260812 01:08:49.444167 21572 ts_tablet_manager.cc:1397] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:49.444191 21576 tablet_bootstrap.cc:492] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: No bootstrap required, opened a new log
I20260812 01:08:49.444232 21576 ts_tablet_manager.cc:1397] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:49.444196 21577 raft_consensus.cc:348] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.444298 21577 raft_consensus.cc:374] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.444341 21577 raft_consensus.cc:729] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba2620bdf2d46119e2181ccdff1c10f, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.444361 21572 raft_consensus.cc:348] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.444396 21577 consensus_queue.cc:260] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.444445 21572 raft_consensus.cc:374] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.444424 21576 raft_consensus.cc:348] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.444481 21572 raft_consensus.cc:729] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c856d245d4a14a31b980a38700296a09, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.444491 21577 ts_tablet_manager.cc:1428] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:49.444497 21576 raft_consensus.cc:374] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.444525 21576 raft_consensus.cc:729] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0392bee06ee742758c136184a537c205, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.444530 21572 consensus_queue.cc:260] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.444569 21576 consensus_queue.cc:260] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.444636 21572 ts_tablet_manager.cc:1428] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:49.444664 21576 ts_tablet_manager.cc:1428] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:49.444635 21582 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.444706 21582 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:49.444728 21583 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.444847 21583 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:49.444826 21581 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.444923 21581 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: This TxnStatusTablet replica's current role is: FOLLOWER
W20260812 01:08:49.453473 21069 tablet.cc:2369] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:08:49.484131 21583 raft_consensus.cc:482] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:49.484251 21583 raft_consensus.cc:504] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.484725 21583 leader_election.cc:288] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331)
I20260812 01:08:49.487743 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.485190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58105 (local address 127.17.130.1:39701)
0812 01:08:49.485239 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.485241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.485249 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:08:49.485287 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.485287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.485295 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.485364 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.485531 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.486027 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.486974 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.487349 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.487441 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.487452 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.487637 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.487682 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:08:49.487736 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.485000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.17.130.2:58105)
0812 01:08:49.485100 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.485118 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.485182 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.485368 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.485369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.485370 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.485454 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.485456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.486120 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.486121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.486870 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.486872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.487442 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.487443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.487539 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.487542 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.487608 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.487656 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:08:49.487848 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.485191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:54863 (local address 127.17.130.3:36331)
0812 01:08:49.485239 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.485241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.485321 (+ 80us) server_negotiation.cc:407] Connection header received
0812 01:08:49.485390 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.485391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.485406 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.485468 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.485638 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.486122 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.487063 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.487433 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.487520 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.487530 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.487706 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.487747 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:08:49.487885 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.485013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36331 (local address 127.17.130.2:54863)
0812 01:08:49.485234 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.485246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.485318 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.485474 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.485475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.485477 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.485557 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.485559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.486221 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.486222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.486966 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.486968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.487541 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.487543 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.487622 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.487626 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.487698 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.487740 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":15,"thread_start_us":124,"threads_started":1}
I20260812 01:08:49.488114 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baeafcd4f8a545b4b9fccd023ab60e02" candidate_uuid: "0392bee06ee742758c136184a537c205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f" is_pre_election: true
I20260812 01:08:49.488281 21003 raft_consensus.cc:2454] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0392bee06ee742758c136184a537c205 in term 0.
I20260812 01:08:49.488335 21457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baeafcd4f8a545b4b9fccd023ab60e02" candidate_uuid: "0392bee06ee742758c136184a537c205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c856d245d4a14a31b980a38700296a09" is_pre_election: true
I20260812 01:08:49.488441 21457 raft_consensus.cc:2454] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0392bee06ee742758c136184a537c205 in term 0.
I20260812 01:08:49.488515 21107 leader_election.cc:302] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0392bee06ee742758c136184a537c205, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:49.488706 21583 raft_consensus.cc:2781] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:49.488765 21583 raft_consensus.cc:482] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:49.488797 21583 raft_consensus.cc:3037] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.489600 21583 raft_consensus.cc:504] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.489776 21583 leader_election.cc:288] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [CANDIDATE]: Term 1 election: Requested vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331)
I20260812 01:08:49.489955 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baeafcd4f8a545b4b9fccd023ab60e02" candidate_uuid: "0392bee06ee742758c136184a537c205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f"
I20260812 01:08:49.490054 21003 raft_consensus.cc:3037] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.490026 21457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "baeafcd4f8a545b4b9fccd023ab60e02" candidate_uuid: "0392bee06ee742758c136184a537c205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c856d245d4a14a31b980a38700296a09"
I20260812 01:08:49.490113 21457 raft_consensus.cc:3037] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.490778 21457 raft_consensus.cc:2454] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0392bee06ee742758c136184a537c205 in term 1.
I20260812 01:08:49.490784 21003 raft_consensus.cc:2454] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0392bee06ee742758c136184a537c205 in term 1.
I20260812 01:08:49.490996 21107 leader_election.cc:302] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0392bee06ee742758c136184a537c205, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:49.491118 21583 raft_consensus.cc:2781] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:49.491318 21583 raft_consensus.cc:686] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [term 1 LEADER]: Becoming Leader. State: Replica: 0392bee06ee742758c136184a537c205, State: Running, Role: LEADER
I20260812 01:08:49.491464 21583 consensus_queue.cc:237] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.491966 21608 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: TxnStatusTablet state changed. Reason: New leader 0392bee06ee742758c136184a537c205. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.492101 21608 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:08:49.492441 21610 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 01:08:49.492563 21610 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 01:08:49.492455 20750 catalog_manager.cc:5495] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0392bee06ee742758c136184a537c205 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } health_report { overall_health: UNKNOWN } } }
I20260812 01:08:49.507936 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.504721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.0.0.1:45470)
0812 01:08:49.504820 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.504835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.504892 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.505123 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.505124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.505126 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:49.505188 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.505191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.505837 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.505838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.506781 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.506784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.507272 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.507278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.507353 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.507377 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.507724 (+ 347us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.507733 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.507790 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.507844 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:08:49.507908 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.504804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.17.130.2:40099)
0812 01:08:49.504865 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.504867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.504877 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:08:49.504984 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.504985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.505002 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.505127 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.505257 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.505757 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.506916 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.507153 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.507243 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.507514 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.507623 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.507696 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.507793 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.507818 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:08:49.510263 21003 raft_consensus.cc:1261] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Refusing update from remote peer 0392bee06ee742758c136184a537c205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:49.510263 21457 raft_consensus.cc:1261] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Refusing update from remote peer 0392bee06ee742758c136184a537c205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:49.510689 21608 consensus_queue.cc:1035] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:49.511235 21583 consensus_queue.cc:1035] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:49.512601 21582 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: TxnStatusTablet state changed. Reason: New leader 0392bee06ee742758c136184a537c205. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.512707 21582 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:49.513906 21581 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: TxnStatusTablet state changed. Reason: New leader 0392bee06ee742758c136184a537c205. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.514008 21581 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:49.513947 21608 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.514067 21608 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:08:49.514201 21582 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.514293 21582 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P eba2620bdf2d46119e2181ccdff1c10f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:49.514449 21614 mvcc.cc:205] Tried to move back new op lower bound from 7317491423270383616 to 7317491423197728768. Current Snapshot: MvccSnapshot[applied={T|T < 7317491423270383616}]
I20260812 01:08:49.514904 21615 mvcc.cc:205] Tried to move back new op lower bound from 7317491423270383616 to 7317491423197728768. Current Snapshot: MvccSnapshot[applied={T|T < 7317491423270383616}]
I20260812 01:08:49.516012 21583 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.516134 21583 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P 0392bee06ee742758c136184a537c205: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:08:49.516777 21581 tablet_replica.cc:439] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0392bee06ee742758c136184a537c205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } } }
I20260812 01:08:49.516978 21581 tablet_replica.cc:442] T baeafcd4f8a545b4b9fccd023ab60e02 P c856d245d4a14a31b980a38700296a09: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:08:49.526927 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.523693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.17.130.1:39701)
0812 01:08:49.523766 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.523768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.523792 (+ 24us) server_negotiation.cc:407] Connection header received
0812 01:08:49.523944 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.523945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.523969 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.524066 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.524266 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.524829 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.525876 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.526210 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.526334 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.526554 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.526675 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.526766 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.526809 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.526833 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:08:49.533315 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.530332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:56063 (local address 127.17.130.2:40099)
0812 01:08:49.530404 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.530406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.530565 (+ 159us) server_negotiation.cc:407] Connection header received
0812 01:08:49.530602 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.530603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.530615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.530684 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.530801 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.531565 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.532453 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.532759 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.532831 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.533004 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.533039 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.533134 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.533207 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.533231 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:08:49.533314 21629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.530222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.17.130.1:56063)
0812 01:08:49.530385 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.530399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.530560 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.530686 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.530686 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.530688 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:49.530746 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.530747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.531653 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.531655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.532357 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.532361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.532839 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.532840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.532892 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.532908 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.533157 (+ 249us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.533159 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.533195 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.533235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260812 01:08:49.536303 21457 raft_consensus.cc:1261] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Refusing update from remote peer eba2620bdf2d46119e2181ccdff1c10f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:49.536443 21218 raft_consensus.cc:1261] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Refusing update from remote peer eba2620bdf2d46119e2181ccdff1c10f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:49.536705 21585 consensus_queue.cc:1035] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:49.536860 21582 consensus_queue.cc:1035] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:49.538844 21619 mvcc.cc:205] Tried to move back new op lower bound from 7317491423378391040 to 7317491422827323392. Current Snapshot: MvccSnapshot[applied={T|T < 7317491423378391040}]
I20260812 01:08:49.546856 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.543650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:36083 (local address 127.17.130.1:39701)
0812 01:08:49.543691 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.543693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.543792 (+ 99us) server_negotiation.cc:407] Connection header received
0812 01:08:49.543814 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.543814 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.543823 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.543873 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.544067 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.544545 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.545919 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.546308 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.546385 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.546619 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.546651 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.546695 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.546772 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.546796 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:08:49.546911 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.543603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.17.130.2:36083)
0812 01:08:49.543678 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.543691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.543758 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.543903 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.543904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.543905 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:49.543998 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.544000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.544624 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.544625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.545737 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.545741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.546410 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.546411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.546495 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.546516 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.546713 (+ 197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.546715 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.546770 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.546820 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 01:08:49.556563 20748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:49.556718 20748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:49.557832 20748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Sending DeleteTablet for 3 replicas of tablet e1c4df153d34459792f318d67c54046d
I20260812 01:08:49.558236 21424 tablet_service.cc:1514] Processing DeleteTablet for tablet e1c4df153d34459792f318d67c54046d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:49 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51604
I20260812 01:08:49.558370 21190 tablet_service.cc:1514] Processing DeleteTablet for tablet e1c4df153d34459792f318d67c54046d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:49 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:45462
I20260812 01:08:49.558596 20972 tablet_service.cc:1514] Processing DeleteTablet for tablet e1c4df153d34459792f318d67c54046d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:49 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:33306
I20260812 01:08:49.558687 21644 tablet_replica.cc:330] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: stopping tablet replica
I20260812 01:08:49.558807 21644 raft_consensus.cc:2229] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:49.558873 21644 raft_consensus.cc:2258] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:49.559343 21644 ts_tablet_manager.cc:1905] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:49.559309 20748 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:08:49.559478 20748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:49.561239 21190 tablet_service.cc:1467] Processing CreateTablet for tablet dec9464670eb48318811b21f57971240 (DEFAULT_TABLE table=test-table [id=70abd20988714a0fa2b22eb162daa386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:49.561295 20972 tablet_service.cc:1467] Processing CreateTablet for tablet dec9464670eb48318811b21f57971240 (DEFAULT_TABLE table=test-table [id=70abd20988714a0fa2b22eb162daa386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:49.561398 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec9464670eb48318811b21f57971240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.561435 20972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec9464670eb48318811b21f57971240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.562664 21576 tablet_bootstrap.cc:492] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Bootstrap starting.
I20260812 01:08:49.563103 21424 tablet_service.cc:1467] Processing CreateTablet for tablet dec9464670eb48318811b21f57971240 (DEFAULT_TABLE table=test-table [id=70abd20988714a0fa2b22eb162daa386]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:49.563249 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec9464670eb48318811b21f57971240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:49.563292 21576 tablet_bootstrap.cc:654] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.563647 21577 tablet_bootstrap.cc:492] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Bootstrap starting.
I20260812 01:08:49.564105 21577 tablet_bootstrap.cc:654] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.564783 21572 tablet_bootstrap.cc:492] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Bootstrap starting.
I20260812 01:08:49.564967 21644 ts_tablet_manager.cc:1918] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:49.565048 21644 log.cc:1196] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/e1c4df153d34459792f318d67c54046d
I20260812 01:08:49.565236 21572 tablet_bootstrap.cc:654] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:49.565367 21644 ts_tablet_manager.cc:1939] T e1c4df153d34459792f318d67c54046d P 0392bee06ee742758c136184a537c205: Deleting consensus metadata
I20260812 01:08:49.565858 20732 catalog_manager.cc:4841] TS 0392bee06ee742758c136184a537c205 (127.17.130.2:40099): tablet e1c4df153d34459792f318d67c54046d (table test-table [id=f986952939c84ab4a5d4041d98a47391]) successfully deleted
I20260812 01:08:49.566795 21577 tablet_bootstrap.cc:492] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: No bootstrap required, opened a new log
I20260812 01:08:49.566855 21577 ts_tablet_manager.cc:1397] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 01:08:49.566892 21572 tablet_bootstrap.cc:492] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: No bootstrap required, opened a new log
I20260812 01:08:49.566942 21572 ts_tablet_manager.cc:1397] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:08:49.567013 21577 raft_consensus.cc:348] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.567088 21577 raft_consensus.cc:374] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.567109 21577 raft_consensus.cc:729] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba2620bdf2d46119e2181ccdff1c10f, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.567122 21572 raft_consensus.cc:348] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.567155 21577 consensus_queue.cc:260] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.567198 21572 raft_consensus.cc:374] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.567220 21572 raft_consensus.cc:729] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c856d245d4a14a31b980a38700296a09, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.567246 21577 ts_tablet_manager.cc:1428] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:49.567267 21572 consensus_queue.cc:260] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.567361 21572 ts_tablet_manager.cc:1428] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:49.568048 21576 tablet_bootstrap.cc:492] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: No bootstrap required, opened a new log
I20260812 01:08:49.568105 21576 ts_tablet_manager.cc:1397] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 01:08:49.568251 21576 raft_consensus.cc:348] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.568328 21576 raft_consensus.cc:374] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:49.568349 21576 raft_consensus.cc:729] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0392bee06ee742758c136184a537c205, State: Initialized, Role: FOLLOWER
I20260812 01:08:49.568389 21576 consensus_queue.cc:260] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.568487 21576 ts_tablet_manager.cc:1428] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:49.568720 21645 tablet_replica.cc:330] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: stopping tablet replica
I20260812 01:08:49.568892 21645 raft_consensus.cc:2229] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:08:49.569104 21645 raft_consensus.cc:2258] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:49.569330 21643 tablet_replica.cc:330] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: stopping tablet replica
I20260812 01:08:49.569490 21643 raft_consensus.cc:2229] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:49.569608 21643 raft_consensus.cc:2258] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:49.569670 21645 ts_tablet_manager.cc:1905] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:49.570475 21643 ts_tablet_manager.cc:1905] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:49.571460 21645 ts_tablet_manager.cc:1918] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:49.571545 21645 log.cc:1196] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/e1c4df153d34459792f318d67c54046d
I20260812 01:08:49.571846 21645 ts_tablet_manager.cc:1939] T e1c4df153d34459792f318d67c54046d P eba2620bdf2d46119e2181ccdff1c10f: Deleting consensus metadata
I20260812 01:08:49.572373 20732 catalog_manager.cc:4841] TS eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701): tablet e1c4df153d34459792f318d67c54046d (table test-table [id=f986952939c84ab4a5d4041d98a47391]) successfully deleted
I20260812 01:08:49.573122 21643 ts_tablet_manager.cc:1918] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:49.573254 21643 log.cc:1196] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/e1c4df153d34459792f318d67c54046d
I20260812 01:08:49.573585 21643 ts_tablet_manager.cc:1939] T e1c4df153d34459792f318d67c54046d P c856d245d4a14a31b980a38700296a09: Deleting consensus metadata
I20260812 01:08:49.574077 20734 catalog_manager.cc:4841] TS c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331): tablet e1c4df153d34459792f318d67c54046d (table test-table [id=f986952939c84ab4a5d4041d98a47391]) successfully deleted
I20260812 01:08:49.634250 21581 raft_consensus.cc:482] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:49.634402 21581 raft_consensus.cc:504] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.634799 21581 leader_election.cc:288] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:49.639194 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.635208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.17.130.3:41279)
0812 01:08:49.635265 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.635280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.635353 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.635707 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.635708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.635710 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.635804 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.635806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.636819 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.636820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.638118 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.638121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.638893 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.638894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.638976 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.638980 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.639045 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.639090 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":2}
I20260812 01:08:49.639740 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.635447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:41279 (local address 127.17.130.1:39701)
0812 01:08:49.635501 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.635503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.635512 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:08:49.635546 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.635547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.635561 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.635612 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.635888 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.636694 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.638226 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.638771 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.638847 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.638858 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.639626 (+ 768us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.639673 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:08:49.639937 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec9464670eb48318811b21f57971240" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f" is_pre_election: true
I20260812 01:08:49.640053 21003 raft_consensus.cc:2454] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 0.
I20260812 01:08:49.640241 21345 leader_election.cc:302] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c856d245d4a14a31b980a38700296a09, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:49.640631 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.635746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:45141 (local address 127.17.130.2:40099)
0812 01:08:49.635793 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.635795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.635994 (+ 199us) server_negotiation.cc:407] Connection header received
0812 01:08:49.636101 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.636103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.636113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.636156 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:08:49.636950 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.637654 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.639679 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.640208 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.640266 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.640275 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.640515 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.640559 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:08:49.640698 21581 raft_consensus.cc:2781] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:49.640745 21581 raft_consensus.cc:482] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:49.640769 21581 raft_consensus.cc:3037] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.640944 21650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.635410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.17.130.3:45141)
0812 01:08:49.635917 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.635929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.636022 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.636226 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.636227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.636229 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:08:49.636844 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.636849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.637771 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.637771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.639431 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.639436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.640346 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.640347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.640428 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.640431 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.640805 (+ 374us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.640858 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":194,"threads_started":1}
I20260812 01:08:49.641287 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec9464670eb48318811b21f57971240" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205" is_pre_election: true
I20260812 01:08:49.641381 21218 raft_consensus.cc:2454] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 0.
I20260812 01:08:49.641803 21581 raft_consensus.cc:504] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.641945 21581 leader_election.cc:288] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 election: Requested vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:49.642189 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec9464670eb48318811b21f57971240" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f"
I20260812 01:08:49.642234 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec9464670eb48318811b21f57971240" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205"
I20260812 01:08:49.642284 21003 raft_consensus.cc:3037] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.642302 21218 raft_consensus.cc:3037] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:49.643206 21218 raft_consensus.cc:2454] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 1.
I20260812 01:08:49.643280 21003 raft_consensus.cc:2454] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 1.
I20260812 01:08:49.643432 21345 leader_election.cc:302] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c856d245d4a14a31b980a38700296a09, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:49.643561 21581 raft_consensus.cc:2781] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:49.643674 21581 raft_consensus.cc:686] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 1 LEADER]: Becoming Leader. State: Replica: c856d245d4a14a31b980a38700296a09, State: Running, Role: LEADER
I20260812 01:08:49.643748 21581 consensus_queue.cc:237] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:49.644537 20748 catalog_manager.cc:5495] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 reported cstate change: term changed from 0 to 1, leader changed from <none> to c856d245d4a14a31b980a38700296a09 (127.17.130.3). New cstate: current_term: 1 leader_uuid: "c856d245d4a14a31b980a38700296a09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } health_report { overall_health: HEALTHY } } }
I20260812 01:08:49.671483 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.667775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.17.130.3:36331)
0812 01:08:49.667843 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.667846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.667858 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:08:49.667914 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.667915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.667948 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.668058 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.668264 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.669030 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.670323 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.670687 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.670777 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.671003 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.671161 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.671250 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.671327 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.671363 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:08:49.679304 21543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.675721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.17.130.3:37823)
0812 01:08:49.675805 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.675819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.675882 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.676093 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.676094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.676095 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:49.676176 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.676178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.677032 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.677033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.677982 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.677986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.678528 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.678530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.678628 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.678655 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.679082 (+ 427us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.679085 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.679151 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.679206 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:08:49.679342 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.675807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:37823 (local address 127.17.130.2:40099)
0812 01:08:49.675853 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.675855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.675865 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:08:49.675977 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.675978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.675990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.676063 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.676255 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.676926 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.678091 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.678409 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.678501 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.678772 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.678812 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.679078 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.679199 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.679253 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:08:49.683256 21003 raft_consensus.cc:1261] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Refusing update from remote peer c856d245d4a14a31b980a38700296a09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:49.683270 21218 raft_consensus.cc:1261] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Refusing update from remote peer c856d245d4a14a31b980a38700296a09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:49.683630 21581 consensus_queue.cc:1035] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:49.683868 21651 consensus_queue.cc:1035] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:49.693928 21566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.690556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:34403 (local address 127.17.130.3:36331)
0812 01:08:49.690630 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:49.690632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:49.690643 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:08:49.690671 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:49.690672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:49.690683 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:49.690744 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:49.691188 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.691839 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.692916 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.693281 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.693379 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.693610 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.693652 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.693727 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:49.693819 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 01:08:49.693843 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:08:49.693993 21306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:49.690259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36331 (local address 127.17.130.2:34403)
0812 01:08:49.690355 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:49.690369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:49.690418 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:49.690806 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:49.690807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:49.690809 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:08:49.690867 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.690869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.692111 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.692111 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.692793 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:49.692796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:49.693391 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:49.693392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:49.693448 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:49.693454 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:08:49.693736 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:08:49.693739 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:08:49.693810 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 01:08:49.693872 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 01:08:49.694669 21619 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 01:08:51.222189 20748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:51.222308 20748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:51.223448 20748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Sending DeleteTablet for 3 replicas of tablet dec9464670eb48318811b21f57971240
I20260812 01:08:51.223793 20972 tablet_service.cc:1514] Processing DeleteTablet for tablet dec9464670eb48318811b21f57971240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:51 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:33306
I20260812 01:08:51.223791 21424 tablet_service.cc:1514] Processing DeleteTablet for tablet dec9464670eb48318811b21f57971240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:51 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51604
I20260812 01:08:51.223869 21190 tablet_service.cc:1514] Processing DeleteTablet for tablet dec9464670eb48318811b21f57971240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:51 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:45462
I20260812 01:08:51.224097 21696 tablet_replica.cc:330] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: stopping tablet replica
I20260812 01:08:51.224182 21696 raft_consensus.cc:2229] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:51.224233 21696 raft_consensus.cc:2258] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:51.224694 21696 ts_tablet_manager.cc:1905] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:51.226547 21695 tablet_replica.cc:330] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: stopping tablet replica
I20260812 01:08:51.226574 21696 ts_tablet_manager.cc:1918] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:08:51.226629 21695 raft_consensus.cc:2229] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:51.226641 21696 log.cc:1196] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/dec9464670eb48318811b21f57971240
I20260812 01:08:51.226681 21695 raft_consensus.cc:2258] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:51.226815 21694 tablet_replica.cc:330] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: stopping tablet replica
I20260812 01:08:51.226876 21694 raft_consensus.cc:2229] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:08:51.226985 21694 raft_consensus.cc:2258] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:51.227010 21695 ts_tablet_manager.cc:1905] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:51.227224 21694 ts_tablet_manager.cc:1905] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:51.228585 21695 ts_tablet_manager.cc:1918] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:08:51.228699 21695 log.cc:1196] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/dec9464670eb48318811b21f57971240
I20260812 01:08:51.228973 21695 ts_tablet_manager.cc:1939] T dec9464670eb48318811b21f57971240 P eba2620bdf2d46119e2181ccdff1c10f: Deleting consensus metadata
I20260812 01:08:51.229254 21694 ts_tablet_manager.cc:1918] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:08:51.229317 21694 log.cc:1196] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/dec9464670eb48318811b21f57971240
I20260812 01:08:51.229413 20732 catalog_manager.cc:4841] TS eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701): tablet dec9464670eb48318811b21f57971240 (table test-table [id=70abd20988714a0fa2b22eb162daa386]) successfully deleted
I20260812 01:08:51.229491 21696 ts_tablet_manager.cc:1939] T dec9464670eb48318811b21f57971240 P 0392bee06ee742758c136184a537c205: Deleting consensus metadata
I20260812 01:08:51.229565 21694 ts_tablet_manager.cc:1939] T dec9464670eb48318811b21f57971240 P c856d245d4a14a31b980a38700296a09: Deleting consensus metadata
I20260812 01:08:51.229846 20732 catalog_manager.cc:4841] TS 0392bee06ee742758c136184a537c205 (127.17.130.2:40099): tablet dec9464670eb48318811b21f57971240 (table test-table [id=70abd20988714a0fa2b22eb162daa386]) successfully deleted
I20260812 01:08:51.229887 20734 catalog_manager.cc:4841] TS c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331): tablet dec9464670eb48318811b21f57971240 (table test-table [id=70abd20988714a0fa2b22eb162daa386]) successfully deleted
Aug 12 01:08:51 dist-test-slave-tfpf krb5kdc[20689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496931, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:08:51 dist-test-slave-tfpf krb5kdc[20689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496931, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:51.257607 21710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:51.245619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.17.130.62:44321)
0812 01:08:51.245896 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:51.245897 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:51.245907 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:08:51.245944 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:51.245945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:51.245956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:51.246025 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:51.246725 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:51.247260 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:51.248149 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:51.248355 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:51.248432 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:51.254613 (+ 6181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:51.254615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:51.254615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:51.254624 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:51.256191 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:51.256583 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:51.256584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:51.256584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:51.256649 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:51.256977 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:51.256978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:51.256978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:51.257140 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:51.257241 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:51.257384 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 01:08:51.257464 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":159,"threads_started":1}
I20260812 01:08:51.259420 20748 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:08:51.259631 20748 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:51.261380 20972 tablet_service.cc:1467] Processing CreateTablet for tablet af4f900d42ce4f4591f40797fcd57a19 (DEFAULT_TABLE table=test-table [id=8e26d1991f1041eaa126a0bce122c096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:51.261402 21424 tablet_service.cc:1467] Processing CreateTablet for tablet af4f900d42ce4f4591f40797fcd57a19 (DEFAULT_TABLE table=test-table [id=8e26d1991f1041eaa126a0bce122c096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:51.261402 21190 tablet_service.cc:1467] Processing CreateTablet for tablet af4f900d42ce4f4591f40797fcd57a19 (DEFAULT_TABLE table=test-table [id=8e26d1991f1041eaa126a0bce122c096]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:51.261561 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4f900d42ce4f4591f40797fcd57a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:51.261569 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4f900d42ce4f4591f40797fcd57a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:51.261756 20972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af4f900d42ce4f4591f40797fcd57a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:51.263111 21711 tablet_bootstrap.cc:492] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Bootstrap starting.
I20260812 01:08:51.263180 21713 tablet_bootstrap.cc:492] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Bootstrap starting.
I20260812 01:08:51.263372 21712 tablet_bootstrap.cc:492] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Bootstrap starting.
I20260812 01:08:51.263643 21711 tablet_bootstrap.cc:654] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:51.263670 21713 tablet_bootstrap.cc:654] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:51.263747 21712 tablet_bootstrap.cc:654] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:51.264302 21712 tablet_bootstrap.cc:492] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: No bootstrap required, opened a new log
I20260812 01:08:51.264317 21713 tablet_bootstrap.cc:492] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: No bootstrap required, opened a new log
I20260812 01:08:51.264372 21712 ts_tablet_manager.cc:1397] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:51.264366 21713 ts_tablet_manager.cc:1397] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:51.264501 21712 raft_consensus.cc:348] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.264494 21713 raft_consensus.cc:348] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.264580 21713 raft_consensus.cc:374] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:51.264580 21712 raft_consensus.cc:374] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:51.264606 21712 raft_consensus.cc:729] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba2620bdf2d46119e2181ccdff1c10f, State: Initialized, Role: FOLLOWER
I20260812 01:08:51.264595 21713 raft_consensus.cc:729] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0392bee06ee742758c136184a537c205, State: Initialized, Role: FOLLOWER
I20260812 01:08:51.264659 21713 consensus_queue.cc:260] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.264664 21712 consensus_queue.cc:260] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.264766 21713 ts_tablet_manager.cc:1428] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:51.264843 21712 ts_tablet_manager.cc:1428] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:08:51.265033 21711 tablet_bootstrap.cc:492] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: No bootstrap required, opened a new log
I20260812 01:08:51.265108 21711 ts_tablet_manager.cc:1397] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:08:51.265215 21711 raft_consensus.cc:348] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.265280 21711 raft_consensus.cc:374] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:51.265311 21711 raft_consensus.cc:729] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c856d245d4a14a31b980a38700296a09, State: Initialized, Role: FOLLOWER
I20260812 01:08:51.265364 21711 consensus_queue.cc:260] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.265478 21711 ts_tablet_manager.cc:1428] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:51.290714 21685 raft_consensus.cc:482] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:51.290833 21685 raft_consensus.cc:504] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.291024 21685 leader_election.cc:288] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:51.291342 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4f900d42ce4f4591f40797fcd57a19" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f" is_pre_election: true
I20260812 01:08:51.291334 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4f900d42ce4f4591f40797fcd57a19" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205" is_pre_election: true
I20260812 01:08:51.291476 21003 raft_consensus.cc:2454] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 0.
I20260812 01:08:51.291479 21218 raft_consensus.cc:2454] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 0.
I20260812 01:08:51.291648 21345 leader_election.cc:302] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0392bee06ee742758c136184a537c205, c856d245d4a14a31b980a38700296a09; no voters:
I20260812 01:08:51.291819 21685 raft_consensus.cc:2781] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:51.291900 21685 raft_consensus.cc:482] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:51.291939 21685 raft_consensus.cc:3037] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:51.292781 21685 raft_consensus.cc:504] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.292937 21685 leader_election.cc:288] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 election: Requested vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:51.293308 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4f900d42ce4f4591f40797fcd57a19" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205"
I20260812 01:08:51.293591 21218 raft_consensus.cc:3037] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:51.293607 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af4f900d42ce4f4591f40797fcd57a19" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f"
I20260812 01:08:51.293692 21003 raft_consensus.cc:3037] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:51.294585 21003 raft_consensus.cc:2454] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 1.
I20260812 01:08:51.294585 21218 raft_consensus.cc:2454] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 1.
I20260812 01:08:51.294788 21345 leader_election.cc:302] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c856d245d4a14a31b980a38700296a09, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:51.294911 21685 raft_consensus.cc:2781] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:51.295011 21685 raft_consensus.cc:686] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 1 LEADER]: Becoming Leader. State: Replica: c856d245d4a14a31b980a38700296a09, State: Running, Role: LEADER
I20260812 01:08:51.295075 21685 consensus_queue.cc:237] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.295847 20750 catalog_manager.cc:5495] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 reported cstate change: term changed from 0 to 1, leader changed from <none> to c856d245d4a14a31b980a38700296a09 (127.17.130.3). New cstate: current_term: 1 leader_uuid: "c856d245d4a14a31b980a38700296a09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } health_report { overall_health: HEALTHY } } }
I20260812 01:08:51.306988 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:51.304298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.17.130.3:36331)
0812 01:08:51.304543 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:51.304545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:51.304554 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:08:51.304584 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:51.304584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:51.304595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:51.304657 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:08:51.304888 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:51.305458 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:51.306259 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:51.306459 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:51.306532 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:51.306692 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:08:51.306720 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:08:51.306761 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:51.306821 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:08:51.306838 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":197,"threads_started":1}
I20260812 01:08:51.309624 21218 raft_consensus.cc:1261] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Refusing update from remote peer c856d245d4a14a31b980a38700296a09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:51.309700 21003 raft_consensus.cc:1261] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Refusing update from remote peer c856d245d4a14a31b980a38700296a09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:51.309855 21685 consensus_queue.cc:1035] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:51.309976 21682 consensus_queue.cc:1035] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:51.325232 20750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:51.325354 20750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:51.326176 20750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Sending DeleteTablet for 3 replicas of tablet af4f900d42ce4f4591f40797fcd57a19
I20260812 01:08:51.326462 21190 tablet_service.cc:1514] Processing DeleteTablet for tablet af4f900d42ce4f4591f40797fcd57a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:51 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:45462
I20260812 01:08:51.326481 21424 tablet_service.cc:1514] Processing DeleteTablet for tablet af4f900d42ce4f4591f40797fcd57a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:51 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51604
I20260812 01:08:51.326571 21696 tablet_replica.cc:330] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: stopping tablet replica
I20260812 01:08:51.326630 21696 raft_consensus.cc:2229] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:51.326601 20972 tablet_service.cc:1514] Processing DeleteTablet for tablet af4f900d42ce4f4591f40797fcd57a19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:51 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:33306
I20260812 01:08:51.326684 21696 raft_consensus.cc:2258] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:51.326713 21694 tablet_replica.cc:330] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: stopping tablet replica
I20260812 01:08:51.326789 21694 raft_consensus.cc:2229] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:08:51.326855 21694 raft_consensus.cc:2258] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:51.326782 20750 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:08:51.326934 20750 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:08:51.326943 21695 tablet_replica.cc:330] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: stopping tablet replica
I20260812 01:08:51.327003 21695 raft_consensus.cc:2229] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:51.327049 21695 raft_consensus.cc:2258] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:51.327062 21696 ts_tablet_manager.cc:1905] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:51.327358 21694 ts_tablet_manager.cc:1905] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:51.327437 21695 ts_tablet_manager.cc:1905] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:51.328897 21696 ts_tablet_manager.cc:1918] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:51.328939 21695 ts_tablet_manager.cc:1918] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:51.328958 21696 log.cc:1196] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/af4f900d42ce4f4591f40797fcd57a19
I20260812 01:08:51.328984 21695 log.cc:1196] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/af4f900d42ce4f4591f40797fcd57a19
I20260812 01:08:51.329003 21694 ts_tablet_manager.cc:1918] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:08:51.329094 21694 log.cc:1196] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/af4f900d42ce4f4591f40797fcd57a19
I20260812 01:08:51.329282 21695 ts_tablet_manager.cc:1939] T af4f900d42ce4f4591f40797fcd57a19 P eba2620bdf2d46119e2181ccdff1c10f: Deleting consensus metadata
I20260812 01:08:51.329316 21694 ts_tablet_manager.cc:1939] T af4f900d42ce4f4591f40797fcd57a19 P c856d245d4a14a31b980a38700296a09: Deleting consensus metadata
I20260812 01:08:51.329317 21696 ts_tablet_manager.cc:1939] T af4f900d42ce4f4591f40797fcd57a19 P 0392bee06ee742758c136184a537c205: Deleting consensus metadata
I20260812 01:08:51.329643 20732 catalog_manager.cc:4841] TS 0392bee06ee742758c136184a537c205 (127.17.130.2:40099): tablet af4f900d42ce4f4591f40797fcd57a19 (table test-table [id=8e26d1991f1041eaa126a0bce122c096]) successfully deleted
I20260812 01:08:51.329730 20732 catalog_manager.cc:4841] TS eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701): tablet af4f900d42ce4f4591f40797fcd57a19 (table test-table [id=8e26d1991f1041eaa126a0bce122c096]) successfully deleted
I20260812 01:08:51.329777 20734 catalog_manager.cc:4841] TS c856d245d4a14a31b980a38700296a09 (127.17.130.3:36331): tablet af4f900d42ce4f4591f40797fcd57a19 (table test-table [id=8e26d1991f1041eaa126a0bce122c096]) successfully deleted
I20260812 01:08:51.330809 20972 tablet_service.cc:1467] Processing CreateTablet for tablet 296de0c68b594cfe93900b69d38c1b13 (DEFAULT_TABLE table=test-table [id=9020861ed80348af80b7d3c621f86488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:51.330861 21424 tablet_service.cc:1467] Processing CreateTablet for tablet 296de0c68b594cfe93900b69d38c1b13 (DEFAULT_TABLE table=test-table [id=9020861ed80348af80b7d3c621f86488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:51.330956 21424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 296de0c68b594cfe93900b69d38c1b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:51.330956 20972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 296de0c68b594cfe93900b69d38c1b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:51.330928 21190 tablet_service.cc:1467] Processing CreateTablet for tablet 296de0c68b594cfe93900b69d38c1b13 (DEFAULT_TABLE table=test-table [id=9020861ed80348af80b7d3c621f86488]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:08:51.331053 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 296de0c68b594cfe93900b69d38c1b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:51.332547 21712 tablet_bootstrap.cc:492] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f: Bootstrap starting.
I20260812 01:08:51.332610 21713 tablet_bootstrap.cc:492] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: Bootstrap starting.
I20260812 01:08:51.332913 21712 tablet_bootstrap.cc:654] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:51.332914 21713 tablet_bootstrap.cc:654] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:51.333163 21711 tablet_bootstrap.cc:492] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Bootstrap starting.
I20260812 01:08:51.333503 21711 tablet_bootstrap.cc:654] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:51.333757 21713 tablet_bootstrap.cc:492] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: No bootstrap required, opened a new log
I20260812 01:08:51.333817 21713 ts_tablet_manager.cc:1397] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 01:08:51.333935 21712 tablet_bootstrap.cc:492] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f: No bootstrap required, opened a new log
I20260812 01:08:51.333993 21712 ts_tablet_manager.cc:1397] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:51.333959 21713 raft_consensus.cc:348] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.334056 21713 raft_consensus.cc:374] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:51.334088 21713 raft_consensus.cc:729] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0392bee06ee742758c136184a537c205, State: Initialized, Role: FOLLOWER
I20260812 01:08:51.334146 21713 consensus_queue.cc:260] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.334159 21712 raft_consensus.cc:348] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.334233 21712 raft_consensus.cc:374] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:51.334265 21712 raft_consensus.cc:729] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba2620bdf2d46119e2181ccdff1c10f, State: Initialized, Role: FOLLOWER
I20260812 01:08:51.334266 21713 ts_tablet_manager.cc:1428] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:08:51.334323 21712 consensus_queue.cc:260] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.334439 21712 ts_tablet_manager.cc:1428] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:51.334872 21711 tablet_bootstrap.cc:492] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: No bootstrap required, opened a new log
I20260812 01:08:51.334918 21711 ts_tablet_manager.cc:1397] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:08:51.335088 21711 raft_consensus.cc:348] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.335150 21711 raft_consensus.cc:374] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:51.335163 21711 raft_consensus.cc:729] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c856d245d4a14a31b980a38700296a09, State: Initialized, Role: FOLLOWER
I20260812 01:08:51.335217 21711 consensus_queue.cc:260] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.335337 21711 ts_tablet_manager.cc:1428] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:51.483832 21685 raft_consensus.cc:482] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:08:51.483942 21685 raft_consensus.cc:504] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.484105 21685 leader_election.cc:288] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:51.484333 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "296de0c68b594cfe93900b69d38c1b13" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f" is_pre_election: true
I20260812 01:08:51.484426 21003 raft_consensus.cc:2454] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 0.
I20260812 01:08:51.484359 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "296de0c68b594cfe93900b69d38c1b13" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205" is_pre_election: true
I20260812 01:08:51.484515 21218 raft_consensus.cc:2454] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 0.
I20260812 01:08:51.484639 21345 leader_election.cc:302] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c856d245d4a14a31b980a38700296a09, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:51.484757 21685 raft_consensus.cc:2781] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:08:51.484793 21685 raft_consensus.cc:482] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:08:51.484818 21685 raft_consensus.cc:3037] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:51.485591 21685 raft_consensus.cc:504] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.485733 21685 leader_election.cc:288] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 election: Requested vote from peers eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701), 0392bee06ee742758c136184a537c205 (127.17.130.2:40099)
I20260812 01:08:51.485918 21003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "296de0c68b594cfe93900b69d38c1b13" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eba2620bdf2d46119e2181ccdff1c10f"
I20260812 01:08:51.485929 21218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "296de0c68b594cfe93900b69d38c1b13" candidate_uuid: "c856d245d4a14a31b980a38700296a09" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0392bee06ee742758c136184a537c205"
I20260812 01:08:51.486007 21003 raft_consensus.cc:3037] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:51.486013 21218 raft_consensus.cc:3037] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:51.486655 21003 raft_consensus.cc:2454] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 1.
I20260812 01:08:51.486675 21218 raft_consensus.cc:2454] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c856d245d4a14a31b980a38700296a09 in term 1.
I20260812 01:08:51.486848 21345 leader_election.cc:302] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c856d245d4a14a31b980a38700296a09, eba2620bdf2d46119e2181ccdff1c10f; no voters:
I20260812 01:08:51.486970 21685 raft_consensus.cc:2781] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:51.487026 21685 raft_consensus.cc:686] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 1 LEADER]: Becoming Leader. State: Replica: c856d245d4a14a31b980a38700296a09, State: Running, Role: LEADER
I20260812 01:08:51.487080 21685 consensus_queue.cc:237] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } }
I20260812 01:08:51.487843 20750 catalog_manager.cc:5495] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 reported cstate change: term changed from 0 to 1, leader changed from <none> to c856d245d4a14a31b980a38700296a09 (127.17.130.3). New cstate: current_term: 1 leader_uuid: "c856d245d4a14a31b980a38700296a09" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c856d245d4a14a31b980a38700296a09" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36331 } health_report { overall_health: HEALTHY } } }
I20260812 01:08:51.524657 21218 raft_consensus.cc:1261] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Refusing update from remote peer c856d245d4a14a31b980a38700296a09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:51.524681 21003 raft_consensus.cc:1261] T 296de0c68b594cfe93900b69d38c1b13 P eba2620bdf2d46119e2181ccdff1c10f [term 1 FOLLOWER]: Refusing update from remote peer c856d245d4a14a31b980a38700296a09: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:08:51.524919 21651 consensus_queue.cc:1035] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eba2620bdf2d46119e2181ccdff1c10f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:08:51.525103 21685 consensus_queue.cc:1035] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0392bee06ee742758c136184a537c205" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 01:08:51.529824 21666 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 01:08:53.062927 20750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:53.063052 20750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:08:53.064270 20750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 331ee5ce8d464b06869522429586d78f: Sending DeleteTablet for 3 replicas of tablet 296de0c68b594cfe93900b69d38c1b13
I20260812 01:08:53.064683 21424 tablet_service.cc:1514] Processing DeleteTablet for tablet 296de0c68b594cfe93900b69d38c1b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:53 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51604
I20260812 01:08:53.064859 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20873
I20260812 01:08:53.064937 20972 tablet_service.cc:1514] Processing DeleteTablet for tablet 296de0c68b594cfe93900b69d38c1b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:53 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:33306
I20260812 01:08:53.064972 21190 tablet_service.cc:1514] Processing DeleteTablet for tablet 296de0c68b594cfe93900b69d38c1b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:08:53 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:45462
I20260812 01:08:53.074179 21770 tablet_replica.cc:330] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: stopping tablet replica
I20260812 01:08:53.074291 21770 raft_consensus.cc:2229] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:08:53.074352 21770 raft_consensus.cc:2258] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:53.074739 21770 ts_tablet_manager.cc:1905] T 296de0c68b594cfe93900b69d38c1b13 P 0392bee06ee742758c136184a537c205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:08:53.077407 21769 tablet_replica.cc:330] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: stopping tablet replica
I20260812 01:08:53.077509 21769 raft_consensus.cc:2229] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:08:53.077607 21769 raft_consensus.cc:2258] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:08:53.078132 21769 ts_tablet_manager.cc:1905] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 01:08:53.085400 20732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:39701 (error 108)
I20260812 01:08:53.085745 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21084
I20260812 01:08:53.097993 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:53.085948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.0.0.1:33326)
0812 01:08:53.097810 (+ 11862us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:53.097911 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11795,"thread_start_us":76,"threads_started":1}
W20260812 01:08:53.098142 20732 catalog_manager.cc:4584] TS eba2620bdf2d46119e2181ccdff1c10f (127.17.130.1:39701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 296de0c68b594cfe93900b69d38c1b13: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39701: connect: Connection refused (error 111)
I20260812 01:08:53.151190 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:53.150939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.0.0.1:33330)
0812 01:08:53.151093 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:53.151144 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 01:08:53.151814 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:53.151668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39701 (local address 127.0.0.1:33344)
0812 01:08:53.151755 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:53.151788 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 01:08:53.162974 21769 ts_tablet_manager.cc:1918] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 01:08:53.163112 21769 log.cc:1196] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/296de0c68b594cfe93900b69d38c1b13
I20260812 01:08:53.163370 21769 ts_tablet_manager.cc:1939] T 296de0c68b594cfe93900b69d38c1b13 P c856d245d4a14a31b980a38700296a09: Deleting consensus metadata
I20260812 01:08:53.168898 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21312
I20260812 01:08:53.168967 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:53.168804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40099 (local address 127.0.0.1:45476)
0812 01:08:53.168878 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:53.168919 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.2:40099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 01:08:53.184453 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20706
2026-08-12T01:08:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6119 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:54 dist-test-slave-tfpf krb5kdc[21777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:54 dist-test-slave-tfpf krb5kdc[21777](info): set up 2 sockets
Aug 12 01:08:54 dist-test-slave-tfpf krb5kdc[21777](info): commencing operation
krb5kdc: starting...
W20260812 01:08:55.936725 17928 mini_kdc.cc:121] Time spent starting KDC: real 2.717s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:08:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:08:56.081645 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40429
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:40429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:56.175658 21799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:56.175904 21799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:56.175928 21799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:56.177431 21799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:08:56.177493 21799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:56.177510 21799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:56.177523 21799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:08:56.177542 21799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:08:56.179272 21799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:40429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:56.179600 21799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:56.179879 21799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:56.186012 21805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.186004 21807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.186004 21804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:56.186527 21799 server_base.cc:1034] running on GCE node
I20260812 01:08:56.186839 21799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:56.187199 21799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:56.188402 21799 hybrid_clock.cc:648] HybridClock initialized: now 1786496936188391 us; error 56 us; skew 500 ppm
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:56.191357 21799 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:08:56.191927 21799 webserver.cc:460] Webserver started at http://127.17.130.62:36291/ using document root <none> and password file <none>
I20260812 01:08:56.192199 21799 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:56.192274 21799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:56.192435 21799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:56.193691 21799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "afa022b3524040899b22780e487ec4c8"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "71e1f947ca4aca695a0f096880567669"
server_key_iv: "06920e01819d57cb800444174f980b3f"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.194096 21799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "afa022b3524040899b22780e487ec4c8"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "71e1f947ca4aca695a0f096880567669"
server_key_iv: "06920e01819d57cb800444174f980b3f"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.195748 21799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:08:56.196867 21816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.197234 21799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:56.197383 21799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "afa022b3524040899b22780e487ec4c8"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "71e1f947ca4aca695a0f096880567669"
server_key_iv: "06920e01819d57cb800444174f980b3f"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.197499 21799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:56.209209 21799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:56.211927 21799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:56.212134 21799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:56.217052 21799 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:40429
I20260812 01:08:56.217164 21892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:40429 every 8 connection(s)
I20260812 01:08:56.217567 21799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:08:56.218788 21893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:56.219110 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21799
I20260812 01:08:56.219218 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:08:56.219427 17928 external_mini_cluster.cc:1428] Setting key 5bcbd36de060e04370252342aa7c5c43
I20260812 01:08:56.222931 21893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8: Bootstrap starting.
I20260812 01:08:56.223673 21893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:56.223992 21893 log.cc:824] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:56.224772 21893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8: No bootstrap required, opened a new log
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:56.227103 21893 raft_consensus.cc:348] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa022b3524040899b22780e487ec4c8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40429 } }
I20260812 01:08:56.227303 21893 raft_consensus.cc:374] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:56.227332 21893 raft_consensus.cc:729] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa022b3524040899b22780e487ec4c8, State: Initialized, Role: FOLLOWER
I20260812 01:08:56.227500 21893 consensus_queue.cc:260] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa022b3524040899b22780e487ec4c8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40429 } }
I20260812 01:08:56.227581 21893 raft_consensus.cc:388] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:08:56.227605 21893 raft_consensus.cc:482] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:08:56.227670 21893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:56.228310 21893 raft_consensus.cc:504] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa022b3524040899b22780e487ec4c8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40429 } }
I20260812 01:08:56.228405 21893 leader_election.cc:302] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afa022b3524040899b22780e487ec4c8; no voters:
I20260812 01:08:56.228647 21893 leader_election.cc:288] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:08:56.228680 21898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:56.228801 21898 raft_consensus.cc:686] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [term 1 LEADER]: Becoming Leader. State: Replica: afa022b3524040899b22780e487ec4c8, State: Running, Role: LEADER
I20260812 01:08:56.228948 21893 sys_catalog.cc:564] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:08:56.228948 21898 consensus_queue.cc:237] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa022b3524040899b22780e487ec4c8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40429 } }
I20260812 01:08:56.229354 21898 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa022b3524040899b22780e487ec4c8. Latest consensus state: current_term: 1 leader_uuid: "afa022b3524040899b22780e487ec4c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa022b3524040899b22780e487ec4c8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40429 } } }
I20260812 01:08:56.229377 21901 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afa022b3524040899b22780e487ec4c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa022b3524040899b22780e487ec4c8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40429 } } }
I20260812 01:08:56.229555 21898 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:56.229568 21901 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:56.229887 21909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:08:56.230636 21909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:08:56.232524 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.220351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.17.130.62:40429)
0812 01:08:56.220569 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:56.220580 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:56.220666 (+ 86us) server_negotiation.cc:407] Connection header received
0812 01:08:56.221244 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:56.221251 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:56.221305 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:56.221459 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:56.222169 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.223165 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.223989 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.224747 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.224921 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.227959 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:56.227964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:56.227964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:56.227978 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:56.230381 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.230993 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.230997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.230997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.231117 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.231364 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.231364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.231365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.231700 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:56.231849 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:56.232141 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 01:08:56.232301 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260812 01:08:56.233557 21909 catalog_manager.cc:1347] Generated new cluster ID: 403ae7678e0045a384237e1176e38a07
I20260812 01:08:56.233624 21909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:08:56.261941 21909 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:08:56.262828 21909 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:08:56.268810 21909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afa022b3524040899b22780e487ec4c8: Generated new TSK 0
I20260812 01:08:56.269137 21909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:08:56.316236 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40429
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:56.428588 21928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:56.428851 21928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:56.428879 21928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:56.430306 21928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:56.430367 21928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:56.430399 21928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:08:56.432030 21928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:56.432269 21928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:56.432475 21928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:56.438524 21944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.438524 21938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.438717 21941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:56.439136 21928 server_base.cc:1034] running on GCE node
I20260812 01:08:56.439404 21928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:56.440001 21928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:56.441239 21928 hybrid_clock.cc:648] HybridClock initialized: now 1786496936441207 us; error 45 us; skew 500 ppm
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:56.458360 21928 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:08:56.459048 21928 webserver.cc:460] Webserver started at http://127.17.130.1:39037/ using document root <none> and password file <none>
I20260812 01:08:56.459576 21928 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:56.459669 21928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:56.459836 21928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:56.461790 21928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "b6f91d1527d24176bc8b22796d2ee041"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "e330837a966027a7782005dd031b2322"
server_key_iv: "1907ff68315cbd389fdc232f4a3e7882"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.462430 21928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "b6f91d1527d24176bc8b22796d2ee041"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "e330837a966027a7782005dd031b2322"
server_key_iv: "1907ff68315cbd389fdc232f4a3e7882"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.464568 21928 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:08:56.466095 21953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.466388 21928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:56.466508 21928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "b6f91d1527d24176bc8b22796d2ee041"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "e330837a966027a7782005dd031b2322"
server_key_iv: "1907ff68315cbd389fdc232f4a3e7882"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.466600 21928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:56.500481 21928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:56.503364 21928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:56.503499 21928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:56.503728 21928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:56.504102 21928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:56.504146 21928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.504172 21928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:56.504185 21928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.511436 21928 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:42947
I20260812 01:08:56.511484 22139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:42947 every 8 connection(s)
I20260812 01:08:56.511881 21928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:08:56.516474 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21928
I20260812 01:08:56.516605 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:08:56.516842 17928 external_mini_cluster.cc:1428] Setting key c91aa950bc4a0d8d520a2ff729310908
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:56.525377 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.513266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:37391 (local address 127.17.130.62:40429)
0812 01:08:56.513373 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:56.513377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:56.513591 (+ 214us) server_negotiation.cc:407] Connection header received
0812 01:08:56.514437 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:56.514439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:56.514454 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:56.514525 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:56.515721 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.516346 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.518098 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.518412 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.518506 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.522199 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:56.522201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:56.522201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:56.522211 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:56.523748 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.524226 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.524227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.524228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.524294 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.524595 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.524596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.524597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.524729 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:56.524806 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:56.525173 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 01:08:56.525274 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 01:08:56.525597 22142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.513129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40429 (local address 127.17.130.1:37391)
0812 01:08:56.513389 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:56.513449 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:56.514146 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:56.514677 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:56.514682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:56.515359 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:56.515587 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:56.515594 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.516481 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.516483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:56.517999 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:56.518003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.518609 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.518611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:56.518868 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.519938 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:56.519953 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:56.522068 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:56.523865 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:56.523874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:56.523877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:56.524135 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:56.524380 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:56.524381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:56.524381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:56.524516 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:56.524810 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:56.524818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:56.524982 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 01:08:56.525128 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 01:08:56.525320 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":90,"threads_started":1}
I20260812 01:08:56.526093 22140 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40429
I20260812 01:08:56.526198 22140 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:56.526428 22140 heartbeater.cc:511] Master 127.17.130.62:40429 requested a full tablet report, sending...
I20260812 01:08:56.527014 21838 ts_manager.cc:194] Registered new tserver with Master: b6f91d1527d24176bc8b22796d2ee041 (127.17.130.1:42947)
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
I20260812 01:08:56.527781 21838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:37391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:08:56.552408 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40429
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:56.653824 22146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:56.654384 22146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:56.654433 22146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:56.656566 22146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:56.656683 22146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:56.656731 22146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:08:56.659603 22146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:56.660230 22146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:56.660816 22146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:56.667472 22159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.667508 22154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.667505 22156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:56.667807 22146 server_base.cc:1034] running on GCE node
I20260812 01:08:56.668071 22146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:56.668365 22146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:56.669571 22146 hybrid_clock.cc:648] HybridClock initialized: now 1786496936669557 us; error 33 us; skew 500 ppm
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:56.672703 22146 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:08:56.673346 22146 webserver.cc:460] Webserver started at http://127.17.130.2:44997/ using document root <none> and password file <none>
I20260812 01:08:56.673633 22146 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:56.673709 22146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:56.673874 22146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:56.675060 22146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "6fca20411f204632919af08ca6aea489"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "83e4d2977b48adbc9ad20e5230306df7"
server_key_iv: "730ffce0c93b6d6b72d5c2c653b6d147"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.675465 22146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "6fca20411f204632919af08ca6aea489"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "83e4d2977b48adbc9ad20e5230306df7"
server_key_iv: "730ffce0c93b6d6b72d5c2c653b6d147"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.677255 22146 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:08:56.678643 22167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.678959 22146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:56.679126 22146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "6fca20411f204632919af08ca6aea489"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "83e4d2977b48adbc9ad20e5230306df7"
server_key_iv: "730ffce0c93b6d6b72d5c2c653b6d147"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.679229 22146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:56.691337 22146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:56.693877 22146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:56.694099 22146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:56.694394 22146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:56.694764 22146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:56.694823 22146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.694867 22146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:56.694897 22146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.704011 22146 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:38545
I20260812 01:08:56.704068 22345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:38545 every 8 connection(s)
I20260812 01:08:56.704448 22146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:08:56.711022 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22146
I20260812 01:08:56.711166 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:08:56.711385 17928 external_mini_cluster.cc:1428] Setting key a9cef8bd51628796b0f824781a1a47dd
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:56.715996 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.705363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:45375 (local address 127.17.130.62:40429)
0812 01:08:56.705427 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:56.705430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:56.705793 (+ 363us) server_negotiation.cc:407] Connection header received
0812 01:08:56.706366 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:56.706368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:56.706380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:56.706445 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:56.707558 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.708090 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.709248 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.709467 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.709554 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.712252 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:56.712253 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:56.712254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:56.712262 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:56.714239 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.714831 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.714833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.714834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.714907 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.715213 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.715215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.715215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.715355 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:56.715435 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:56.715789 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 01:08:56.715893 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:08:56.716240 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.705389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40429 (local address 127.17.130.2:45375)
0812 01:08:56.705639 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:56.705680 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:56.706183 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:56.706566 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:56.706571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:56.707233 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:56.707446 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:56.707451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.708199 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.708200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:56.709146 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:56.709149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.709578 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.709579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:56.709736 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.710344 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:56.710354 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:56.712098 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:56.714400 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:56.714410 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:56.714413 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:56.714716 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:56.714990 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:56.714991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:56.714991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:56.715135 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:56.715438 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:56.715447 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:56.715630 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 01:08:56.715752 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 01:08:56.715956 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":111,"threads_started":1}
I20260812 01:08:56.717052 22347 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40429
I20260812 01:08:56.717223 22347 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:56.717442 22347 heartbeater.cc:511] Master 127.17.130.62:40429 requested a full tablet report, sending...
I20260812 01:08:56.717964 21838 ts_manager.cc:194] Registered new tserver with Master: 6fca20411f204632919af08ca6aea489 (127.17.130.2:38545)
I20260812 01:08:56.718398 21838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:45375
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:08:56.745500 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40429
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:56.837718 22353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:56.837950 22353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:56.837977 22353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:56.839223 22353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:56.839279 22353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:56.839308 22353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:08:56.840911 22353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:56.841187 22353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:56.841473 22353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:56.847473 22359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:56.847435 22362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:56.847528 22353 server_base.cc:1034] running on GCE node
W20260812 01:08:56.847708 22360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:56.847992 22353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:56.848237 22353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:56.849400 22353 hybrid_clock.cc:648] HybridClock initialized: now 1786496936849334 us; error 80 us; skew 500 ppm
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:56.852444 22353 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:08:56.853142 22353 webserver.cc:460] Webserver started at http://127.17.130.3:41315/ using document root <none> and password file <none>
I20260812 01:08:56.853446 22353 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:56.853499 22353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:56.853654 22353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:56.855020 22353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "9dc9f4749ff34cbb805d829d918238cd"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "ddeadac4df3bd35ecf87c576b9c241f3"
server_key_iv: "848ccbb1a1392fe248976dc4d0310c3b"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.855499 22353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "9dc9f4749ff34cbb805d829d918238cd"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "ddeadac4df3bd35ecf87c576b9c241f3"
server_key_iv: "848ccbb1a1392fe248976dc4d0310c3b"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.857241 22353 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:08:56.858688 22370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.859064 22353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:56.859309 22353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "9dc9f4749ff34cbb805d829d918238cd"
format_stamp: "Formatted at 2026-08-12 01:08:56 on dist-test-slave-tfpf"
server_key: "ddeadac4df3bd35ecf87c576b9c241f3"
server_key_iv: "848ccbb1a1392fe248976dc4d0310c3b"
server_key_version: "encryptionkey@0"
I20260812 01:08:56.859411 22353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:56.880609 22353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:56.884231 22353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:56.884405 22353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:56.884697 22353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:08:56.885088 22353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:08:56.885136 22353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.885172 22353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:08:56.885201 22353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:56.892200 22353 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:40221
I20260812 01:08:56.892289 22538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:40221 every 8 connection(s)
I20260812 01:08:56.892655 22353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:08:56.893330 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22353
I20260812 01:08:56.893451 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:08:56.893690 17928 external_mini_cluster.cc:1428] Setting key f7c0f0eef511f974e5adef5c93e86bd9
Aug 12 01:08:56 dist-test-slave-tfpf krb5kdc[21777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496936, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:56.906672 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.894620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:38343 (local address 127.17.130.62:40429)
0812 01:08:56.894672 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:56.894674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:56.894942 (+ 268us) server_negotiation.cc:407] Connection header received
0812 01:08:56.895599 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:56.895600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:56.895615 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:56.895686 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:56.896850 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.897663 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.898926 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.899179 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.899270 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.903069 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:56.903071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:56.903072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:56.903081 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:56.904688 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.905382 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.905383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.905383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.905460 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:56.905841 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:56.905842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:56.905842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:56.906045 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:56.906128 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:56.906518 (+ 390us) server_negotiation.cc:299] Negotiation successful
0812 01:08:56.906600 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:08:56.906898 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:56.894568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40429 (local address 127.17.130.3:38343)
0812 01:08:56.894797 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 01:08:56.894840 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:08:56.895453 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:08:56.895812 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:08:56.895816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:08:56.896444 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:08:56.896687 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:56.896692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.897829 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.897830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:56.898787 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:08:56.898790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:08:56.899387 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:08:56.899389 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:08:56.899609 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:56.900612 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:08:56.900623 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:08:56.902923 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:08:56.904809 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:56.904818 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:56.904820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:56.905242 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:56.905565 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:08:56.905566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:08:56.905567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:08:56.905759 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:08:56.906156 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:08:56.906164 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:08:56.906363 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 01:08:56.906473 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 01:08:56.906647 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":142,"threads_started":1}
I20260812 01:08:56.907454 22540 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40429
I20260812 01:08:56.907573 22540 heartbeater.cc:464] Registering TS with master...
I20260812 01:08:56.907789 22540 heartbeater.cc:511] Master 127.17.130.62:40429 requested a full tablet report, sending...
I20260812 01:08:56.908279 21838 ts_manager.cc:194] Registered new tserver with Master: 9dc9f4749ff34cbb805d829d918238cd (127.17.130.3:40221)
I20260812 01:08:56.908706 21838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:38343
I20260812 01:08:56.917639 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 01:08:56.933393 21896 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:08:56.931620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.17.130.62:40429)
0812 01:08:56.931698 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:56.931700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:56.931859 (+ 159us) server_negotiation.cc:407] Connection header received
0812 01:08:56.931900 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:56.931901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:56.931915 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:56.932277 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:56.933308 (+ 1031us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37034: BlockingRecv error: recv got EOF from 127.0.0.1:37034 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:08:56.934368 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21928
I20260812 01:08:56.968152 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22146
I20260812 01:08:57.002538 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22353
I20260812 01:08:57.010934 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21799
2026-08-12T01:08:57Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3820 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:57 dist-test-slave-tfpf krb5kdc[22562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:57 dist-test-slave-tfpf krb5kdc[22562](info): set up 2 sockets
Aug 12 01:08:57 dist-test-slave-tfpf krb5kdc[22562](info): commencing operation
krb5kdc: starting...
W20260812 01:08:58.340641 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:08:58 dist-test-slave-tfpf krb5kdc[22562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:08:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:08:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:08:58.486994 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:36091
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:08:58.580313 22584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:08:58.580595 22584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:08:58.580633 22584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:08:58.581885 22584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:08:58.581974 22584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:08:58.582000 22584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:08:58.582020 22584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:08:58.582059 22584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:08:58.583668 22584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:36091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:36091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:08:58.583981 22584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:08:58.584244 22584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:08:58.589462 22591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:58.589660 22592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:08:58.589673 22595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:08:58.590665 22584 server_base.cc:1034] running on GCE node
I20260812 01:08:58.591019 22584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:08:58.591359 22584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:08:58.592547 22584 hybrid_clock.cc:648] HybridClock initialized: now 1786496938592510 us; error 52 us; skew 500 ppm
Aug 12 01:08:58 dist-test-slave-tfpf krb5kdc[22562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496938, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:08:58.595798 22584 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:08:58.596381 22584 webserver.cc:460] Webserver started at http://127.17.130.62:38845/ using document root <none> and password file <none>
I20260812 01:08:58.596665 22584 fs_manager.cc:359] Metadata directory not provided
I20260812 01:08:58.596757 22584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:08:58.596928 22584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:08:58.598258 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4"
format_stamp: "Formatted at 2026-08-12 01:08:58 on dist-test-slave-tfpf"
server_key: "dcc3925dc32b09da40227f372dd320c2"
server_key_iv: "3a7e1248c66e61530ee4da48dba96a1e"
server_key_version: "encryptionkey@0"
I20260812 01:08:58.598691 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4"
format_stamp: "Formatted at 2026-08-12 01:08:58 on dist-test-slave-tfpf"
server_key: "dcc3925dc32b09da40227f372dd320c2"
server_key_iv: "3a7e1248c66e61530ee4da48dba96a1e"
server_key_version: "encryptionkey@0"
I20260812 01:08:58.600466 22584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:08:58.601543 22605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:08:58.601809 22584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:08:58.601955 22584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4"
format_stamp: "Formatted at 2026-08-12 01:08:58 on dist-test-slave-tfpf"
server_key: "dcc3925dc32b09da40227f372dd320c2"
server_key_iv: "3a7e1248c66e61530ee4da48dba96a1e"
server_key_version: "encryptionkey@0"
I20260812 01:08:58.602052 22584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:08:58.621502 22584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:08:58.624208 22584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:08:58.624418 22584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:08:58.629837 22584 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:36091
I20260812 01:08:58.629937 22686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:36091 every 8 connection(s)
I20260812 01:08:58.630661 22584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:08:58.631494 22687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:08:58.634728 22687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4: Bootstrap starting.
I20260812 01:08:58.634750 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22584
I20260812 01:08:58.634857 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:08:58.635072 17928 external_mini_cluster.cc:1428] Setting key f6e9b877e90123f06a08551d07f90ae8
I20260812 01:08:58.635628 22687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4: Neither blocks nor log segments found. Creating new log.
I20260812 01:08:58.636060 22687 log.cc:824] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4: Log is configured to *not* fsync() on all Append() calls
I20260812 01:08:58.636827 22687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4: No bootstrap required, opened a new log
I20260812 01:08:58.638983 22687 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36091 } }
I20260812 01:08:58.639179 22687 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:08:58.639237 22687 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0a3d5ac9a1a4fe198adc1592cdd52e4, State: Initialized, Role: FOLLOWER
I20260812 01:08:58.639451 22687 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36091 } }
I20260812 01:08:58.639550 22687 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:08:58.639583 22687 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:08:58.639622 22687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:08:58.640465 22687 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36091 } }
I20260812 01:08:58.640571 22687 leader_election.cc:302] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0a3d5ac9a1a4fe198adc1592cdd52e4; no voters:
I20260812 01:08:58.640802 22687 leader_election.cc:288] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:08:58.640961 22694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:08:58.641155 22694 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [term 1 LEADER]: Becoming Leader. State: Replica: c0a3d5ac9a1a4fe198adc1592cdd52e4, State: Running, Role: LEADER
I20260812 01:08:58.641161 22687 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:08:58.641382 22694 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36091 } }
I20260812 01:08:58.641909 22696 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0a3d5ac9a1a4fe198adc1592cdd52e4. Latest consensus state: current_term: 1 leader_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36091 } } }
I20260812 01:08:58.641939 22698 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0a3d5ac9a1a4fe198adc1592cdd52e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36091 } } }
I20260812 01:08:58.642063 22698 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:58.641994 22696 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4 [sys.catalog]: This master's current role is: LEADER
I20260812 01:08:58.642400 22710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 01:08:58 dist-test-slave-tfpf krb5kdc[22562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:08:58.643800 22710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:08:58.647102 22710 catalog_manager.cc:1347] Generated new cluster ID: 5ed8ba52a079451d9c1f9f082fe086b1
I20260812 01:08:58.647178 22710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:08:58.648880 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:58.635690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48342 (local address 127.17.130.62:36091)
0812 01:08:58.635928 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:58.635939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:58.636037 (+ 98us) server_negotiation.cc:407] Connection header received
0812 01:08:58.636611 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:58.636618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:58.636684 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:58.636854 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:58.637565 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:58.638576 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:58.639413 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:58.640044 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:58.640184 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:58.644339 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:58.644343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:58.644344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:08:58.644359 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:58.646760 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:58.647302 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:58.647306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:58.647307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:58.647433 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:08:58.647708 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:08:58.647710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:08:58.647710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:08:58.648054 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:58.648214 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:58.648449 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 01:08:58.648624 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":98,"threads_started":1}
I20260812 01:08:58.657204 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:08:58.661267 22710 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:08:58.661886 22710 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:08:58.669023 22710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0a3d5ac9a1a4fe198adc1592cdd52e4: Generated new TSK 0
I20260812 01:08:58.669265 22710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:08:58.679489 22691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:08:58.673476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48348 (local address 127.17.130.62:36091)
0812 01:08:58.673570 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 01:08:58.673573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:08:58.673791 (+ 218us) server_negotiation.cc:407] Connection header received
0812 01:08:58.673859 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:08:58.673861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:08:58.673876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:08:58.673987 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:08:58.674635 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:58.675504 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:58.676421 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:08:58.677536 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:08:58.677626 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:08:58.678487 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:08:58.678489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:08:58.678489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:08:58.678748 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:08:58.678770 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:08:58.678777 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:08:58.678851 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:08:58.679351 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 01:08:58.679389 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 01:08:58.683017 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22584
2026-08-12T01:08:58Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1669 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:08:58 dist-test-slave-tfpf krb5kdc[22734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:08:58 dist-test-slave-tfpf krb5kdc[22734](info): set up 2 sockets
Aug 12 01:08:58 dist-test-slave-tfpf krb5kdc[22734](info): commencing operation
krb5kdc: starting...
W20260812 01:09:00.045035 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:00.201685 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:37431
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:37431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:00.292497 22753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:00.292799 22753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:00.292842 22753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:00.294718 22753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:00.294813 22753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:00.294840 22753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:00.294864 22753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:00.294893 22753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:00.296886 22753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:37431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:37431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:00.297252 22753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:00.297515 22753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:00.303033 22761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:00.303009 22764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:00.303257 22753 server_base.cc:1034] running on GCE node
W20260812 01:09:00.303369 22762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:00.303727 22753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:00.304070 22753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:00.305249 22753 hybrid_clock.cc:648] HybridClock initialized: now 1786496940305216 us; error 57 us; skew 500 ppm
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:00.308197 22753 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:00.308774 22753 webserver.cc:460] Webserver started at http://127.17.130.62:40401/ using document root <none> and password file <none>
I20260812 01:09:00.309027 22753 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:00.309139 22753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:00.309293 22753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:00.310446 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "1cde7a3147924e6dba70cf41f69a7c62"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "b8ed0fd91dd0f1eade50e3271df114ca"
server_key_iv: "87424606f113b4bf8e6147c4582fbb2d"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.310856 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "1cde7a3147924e6dba70cf41f69a7c62"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "b8ed0fd91dd0f1eade50e3271df114ca"
server_key_iv: "87424606f113b4bf8e6147c4582fbb2d"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.312438 22753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:00.313465 22771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.313750 22753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:00.313887 22753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "1cde7a3147924e6dba70cf41f69a7c62"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "b8ed0fd91dd0f1eade50e3271df114ca"
server_key_iv: "87424606f113b4bf8e6147c4582fbb2d"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.313992 22753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:00.329792 22753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:00.332335 22753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:00.332548 22753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:00.338778 22753 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:37431
I20260812 01:09:00.338852 22852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:37431 every 8 connection(s)
I20260812 01:09:00.339325 22753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:00.340497 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:00.343879 22853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62: Bootstrap starting.
I20260812 01:09:00.344591 22853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:00.344909 22853 log.cc:824] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:00.345768 22853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62: No bootstrap required, opened a new log
I20260812 01:09:00.347529 22853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7a3147924e6dba70cf41f69a7c62" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37431 } }
I20260812 01:09:00.347679 22853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:00.347707 22853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cde7a3147924e6dba70cf41f69a7c62, State: Initialized, Role: FOLLOWER
I20260812 01:09:00.348063 22853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7a3147924e6dba70cf41f69a7c62" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37431 } }
I20260812 01:09:00.348304 22853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:00.348342 22853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:00.348456 22853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:00.349427 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22753
I20260812 01:09:00.349579 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:00.349766 17928 external_mini_cluster.cc:1428] Setting key 92c725f337fadbc0f47ac90d37db3ee0
I20260812 01:09:00.350175 22853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7a3147924e6dba70cf41f69a7c62" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37431 } }
I20260812 01:09:00.350337 22853 leader_election.cc:302] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cde7a3147924e6dba70cf41f69a7c62; no voters:
I20260812 01:09:00.350541 22853 leader_election.cc:288] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:00.350615 22858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:00.350744 22858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [term 1 LEADER]: Becoming Leader. State: Replica: 1cde7a3147924e6dba70cf41f69a7c62, State: Running, Role: LEADER
I20260812 01:09:00.350853 22853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:00.350867 22858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7a3147924e6dba70cf41f69a7c62" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37431 } }
I20260812 01:09:00.351269 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cde7a3147924e6dba70cf41f69a7c62. Latest consensus state: current_term: 1 leader_uuid: "1cde7a3147924e6dba70cf41f69a7c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7a3147924e6dba70cf41f69a7c62" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37431 } } }
I20260812 01:09:00.351357 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:00.351301 22860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cde7a3147924e6dba70cf41f69a7c62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7a3147924e6dba70cf41f69a7c62" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37431 } } }
I20260812 01:09:00.351540 22860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:00.351845 22867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:00.352680 22867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:00.355644 22867 catalog_manager.cc:1347] Generated new cluster ID: fd7df0ee0f7540daba0bd3bebd95d85d
I20260812 01:09:00.355717 22867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:00.364899 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.350365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.17.130.62:37431)
0812 01:09:00.350691 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:00.350704 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:00.350749 (+ 45us) server_negotiation.cc:407] Connection header received
0812 01:09:00.350899 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:00.350906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:00.351070 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:00.351260 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:00.352245 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.354553 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.356373 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.357196 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.357311 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.360685 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:00.360690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:00.360691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:00.360709 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:00.363011 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.363466 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.363470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.363471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.363610 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.363837 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.363838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.363838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.364210 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:00.364380 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:00.364481 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 01:09:00.364647 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":155,"threads_started":1}
I20260812 01:09:00.383829 22867 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:00.384610 22867 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:00.395860 22867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cde7a3147924e6dba70cf41f69a7c62: Generated new TSK 0
I20260812 01:09:00.396317 22867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:09:00.431953 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37431
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:00.529686 22893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:00.529981 22893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:00.530035 22893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:00.531325 22893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:00.531410 22893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:00.531461 22893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:00.533638 22893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:00.534189 22893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:00.534559 22893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:00.542340 22903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:00.542364 22900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:00.542367 22899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:00.543071 22893 server_base.cc:1034] running on GCE node
I20260812 01:09:00.543377 22893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:00.543709 22893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:00.544987 22893 hybrid_clock.cc:648] HybridClock initialized: now 1786496940544960 us; error 40 us; skew 500 ppm
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:00.549293 22893 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:09:00.549898 22893 webserver.cc:460] Webserver started at http://127.17.130.1:34733/ using document root <none> and password file <none>
I20260812 01:09:00.550171 22893 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:00.550247 22893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:00.550424 22893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:00.551723 22893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "b1fea0072fe848318ca2e7c2d7cc27b3"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "b4a5c156b7639e522a65f5c4e238eae6"
server_key_iv: "ca1a4fc0b3f901e9b871e8c530c920f1"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.552134 22893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "b1fea0072fe848318ca2e7c2d7cc27b3"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "b4a5c156b7639e522a65f5c4e238eae6"
server_key_iv: "ca1a4fc0b3f901e9b871e8c530c920f1"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.553752 22893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 01:09:00.554737 22914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.555085 22893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:00.555238 22893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "b1fea0072fe848318ca2e7c2d7cc27b3"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "b4a5c156b7639e522a65f5c4e238eae6"
server_key_iv: "ca1a4fc0b3f901e9b871e8c530c920f1"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.555332 22893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:00.580261 22893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:00.584443 22893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:00.584784 22893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:00.585188 22893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:00.585561 22893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:00.585621 22893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.585672 22893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:00.585701 22893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.592897 22893 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:33843
I20260812 01:09:00.593053 23086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:33843 every 8 connection(s)
I20260812 01:09:00.593472 22893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:00.600672 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22893
I20260812 01:09:00.600849 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:00.601166 17928 external_mini_cluster.cc:1428] Setting key 9e8feb7c9d49b478004fdfeec812c0cc
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:00.607170 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.595131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47337 (local address 127.17.130.62:37431)
0812 01:09:00.595196 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:00.595199 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:00.595440 (+ 241us) server_negotiation.cc:407] Connection header received
0812 01:09:00.596003 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:00.596004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:00.596020 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:00.596084 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:00.597428 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.598137 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.599783 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.600102 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.600218 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.603200 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:00.603202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:00.603202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:00.603224 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:00.604830 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.605736 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.605738 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.605738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.605829 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.606284 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.606285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.606286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.606462 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:00.606524 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:00.606885 (+ 361us) server_negotiation.cc:299] Negotiation successful
0812 01:09:00.607011 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:09:00.607198 23091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.594888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:37431 (local address 127.17.130.1:47337)
0812 01:09:00.595273 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:00.595317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:00.595875 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:00.596144 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:00.596147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:00.596958 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:00.597297 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:00.597304 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.598396 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.598398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:00.599593 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:00.599601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.600264 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.600266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:00.600425 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.601161 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:00.601174 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:00.603037 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:00.605054 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:00.605163 (+ 109us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:00.605170 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:00.605601 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:00.605978 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:00.605980 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:00.605980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:00.606176 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:00.606538 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:00.606546 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:00.606712 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 01:09:00.606812 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 01:09:00.606984 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":241,"threads_started":1}
I20260812 01:09:00.607894 23087 heartbeater.cc:347] Connected to a master server at 127.17.130.62:37431
I20260812 01:09:00.608028 23087 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:00.608318 23087 heartbeater.cc:511] Master 127.17.130.62:37431 requested a full tablet report, sending...
I20260812 01:09:00.609170 22794 ts_manager.cc:194] Registered new tserver with Master: b1fea0072fe848318ca2e7c2d7cc27b3 (127.17.130.1:33843)
I20260812 01:09:00.609866 22794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:47337
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:09:00.642158 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37431
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:00.738808 23095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:00.739048 23095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:00.739075 23095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:00.740365 23095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:00.740418 23095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:00.740451 23095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:00.742110 23095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:00.742408 23095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:00.742697 23095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:00.749776 23104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:00.749763 23105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:00.749925 23095 server_base.cc:1034] running on GCE node
W20260812 01:09:00.749763 23108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:00.750358 23095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:00.750682 23095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:00.751930 23095 hybrid_clock.cc:648] HybridClock initialized: now 1786496940751932 us; error 54 us; skew 500 ppm
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:00.755527 23095 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:09:00.756148 23095 webserver.cc:460] Webserver started at http://127.17.130.2:36087/ using document root <none> and password file <none>
I20260812 01:09:00.756403 23095 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:00.756477 23095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:00.756644 23095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:00.758371 23095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "a3df61fd988142a993107d19bff3ca05"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "f93ef212fc6cd1239ff52a4800888774"
server_key_iv: "1669bd65122cb547d742066775a22762"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.758875 23095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "a3df61fd988142a993107d19bff3ca05"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "f93ef212fc6cd1239ff52a4800888774"
server_key_iv: "1669bd65122cb547d742066775a22762"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.760466 23095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:00.761466 23115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.761775 23095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:00.761904 23095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "a3df61fd988142a993107d19bff3ca05"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "f93ef212fc6cd1239ff52a4800888774"
server_key_iv: "1669bd65122cb547d742066775a22762"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.762028 23095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:00.776784 23095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:00.781981 23095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:00.782217 23095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:00.782508 23095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:00.782907 23095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:00.782965 23095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.783010 23095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:00.783043 23095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.790194 23095 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:45851
I20260812 01:09:00.790307 23296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:45851 every 8 connection(s)
I20260812 01:09:00.790720 23095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:00.790896 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 23095
I20260812 01:09:00.791036 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:00.791286 17928 external_mini_cluster.cc:1428] Setting key d314d838d646fb09b5df00622aa2ad5e
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:09:00.804035 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.791947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:45875 (local address 127.17.130.62:37431)
0812 01:09:00.792029 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:00.792032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:00.792434 (+ 402us) server_negotiation.cc:407] Connection header received
0812 01:09:00.793025 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:00.793026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:00.793040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:00.793136 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:00.794242 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.794828 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.796474 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.796826 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.796920 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.800194 (+ 3274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:00.800196 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:00.800197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:00.800208 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:00.802171 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.802682 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.802684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.802684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.802769 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.803247 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.803249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.803249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.803411 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:00.803503 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:00.803875 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 01:09:00.803962 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:09:00.804283 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.791848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:37431 (local address 127.17.130.2:45875)
0812 01:09:00.792276 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:00.792332 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:00.792900 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:00.793239 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:00.793248 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:00.793885 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:00.794120 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:00.794126 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.795019 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.795021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:00.796336 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:00.796341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.797002 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.797004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:00.797231 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.797883 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:00.797893 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:00.800069 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:00.802297 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:00.802311 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:00.802314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:00.802589 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:00.802884 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:00.802885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:00.802886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:00.803055 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:00.803552 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:00.803561 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:00.803722 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 01:09:00.803826 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 01:09:00.803990 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":121,"threads_started":1}
I20260812 01:09:00.804976 23297 heartbeater.cc:347] Connected to a master server at 127.17.130.62:37431
I20260812 01:09:00.805160 23297 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:00.805392 23297 heartbeater.cc:511] Master 127.17.130.62:37431 requested a full tablet report, sending...
I20260812 01:09:00.805918 22794 ts_manager.cc:194] Registered new tserver with Master: a3df61fd988142a993107d19bff3ca05 (127.17.130.2:45851)
I20260812 01:09:00.806314 22794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:45875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:09:00.824586 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37431
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:00.925349 23304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:00.925668 23304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:00.925891 23304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:00.927927 23304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:00.927980 23304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:00.928016 23304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:00.929852 23304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:00.930137 23304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:00.930351 23304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:00.936849 23313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:00.936914 23316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:00.936968 23311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:00.937489 23304 server_base.cc:1034] running on GCE node
I20260812 01:09:00.937774 23304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:00.938083 23304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:00.939332 23304 hybrid_clock.cc:648] HybridClock initialized: now 1786496940939320 us; error 54 us; skew 500 ppm
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:00.943079 23304 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:09:00.943701 23304 webserver.cc:460] Webserver started at http://127.17.130.3:41973/ using document root <none> and password file <none>
I20260812 01:09:00.944003 23304 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:00.944084 23304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:00.944265 23304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:00.945655 23304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "591f71b97f094294ac69194137c0ba49"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "34c3be4c140ed58a6fa70b0d69c5d0bf"
server_key_iv: "ad963237c21f1209dcd7d0b04bfd0eb8"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.946090 23304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "591f71b97f094294ac69194137c0ba49"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "34c3be4c140ed58a6fa70b0d69c5d0bf"
server_key_iv: "ad963237c21f1209dcd7d0b04bfd0eb8"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.947711 23304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:00.948731 23325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.949163 23304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:00.949290 23304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "591f71b97f094294ac69194137c0ba49"
format_stamp: "Formatted at 2026-08-12 01:09:00 on dist-test-slave-tfpf"
server_key: "34c3be4c140ed58a6fa70b0d69c5d0bf"
server_key_iv: "ad963237c21f1209dcd7d0b04bfd0eb8"
server_key_version: "encryptionkey@0"
I20260812 01:09:00.949385 23304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:00.958592 23304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:00.961498 23304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:00.961678 23304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:00.961937 23304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:00.962309 23304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:00.962368 23304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.962412 23304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:00.962445 23304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:00.970474 23304 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:44061
I20260812 01:09:00.970505 23504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:44061 every 8 connection(s)
I20260812 01:09:00.970994 23304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:00.971697 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 23304
I20260812 01:09:00.971796 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:00.972033 17928 external_mini_cluster.cc:1428] Setting key 1ee994663e24ffa0458d212743effa95
Aug 12 01:09:00 dist-test-slave-tfpf krb5kdc[22734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496940, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:00.986043 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.973237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:36285 (local address 127.17.130.62:37431)
0812 01:09:00.973548 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:00.973550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:00.973560 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:00.973588 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:00.973589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:00.973601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:00.973660 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:00.975063 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.976636 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.977975 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.978241 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.978372 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.981611 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:00.981613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:00.981614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:00.981625 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:00.983640 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.984433 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.984437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.984437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.984672 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:00.985188 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:00.985189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:00.985190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:00.985362 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:00.985458 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:00.985825 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 01:09:00.985928 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282}
I20260812 01:09:00.986312 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:00.971903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:37431 (local address 127.17.130.3:36285)
0812 01:09:00.972301 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:00.972352 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:00.973014 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:00.973809 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:00.973814 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:00.974615 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:00.974917 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:00.974926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.976836 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.976837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:00.977801 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:00.977804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:00.978402 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:00.978404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:00.978537 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:00.979225 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:00.979235 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:00.981469 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:00.983767 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:00.983775 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:00.983778 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:00.984070 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:00.984817 (+ 747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:00.984818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:00.984819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:00.985019 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:00.985456 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:00.985465 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:00.985639 (+ 174us) client_negotiation.cc:769] Sending connection context
0812 01:09:00.985786 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 01:09:00.986028 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":94,"threads_started":1}
I20260812 01:09:00.987337 23506 heartbeater.cc:347] Connected to a master server at 127.17.130.62:37431
I20260812 01:09:00.987481 23506 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:00.987739 23506 heartbeater.cc:511] Master 127.17.130.62:37431 requested a full tablet report, sending...
I20260812 01:09:00.988438 22794 ts_manager.cc:194] Registered new tserver with Master: 591f71b97f094294ac69194137c0ba49 (127.17.130.3:44061)
I20260812 01:09:00.988833 22794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:36285
I20260812 01:09:00.996919 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:09:01 dist-test-slave-tfpf krb5kdc[22734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496941, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 01:09:01 dist-test-slave-tfpf krb5kdc[22734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496941, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:01.027518 22857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:01.019241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.17.130.62:37431)
0812 01:09:01.019403 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:01.019406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:01.019703 (+ 297us) server_negotiation.cc:407] Connection header received
0812 01:09:01.019830 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:01.019831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:01.019851 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:01.019917 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:01.020570 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:01.021140 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:01.022023 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:01.022230 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:01.022336 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:01.024799 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:01.024800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:01.024801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:01.024809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:01.026245 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:01.026633 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:01.026634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:01.026634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:01.026695 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:01.026904 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:01.026905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:01.026905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:01.027072 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:01.027191 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:01.027312 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 01:09:01.027398 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260812 01:09:01.027855 22794 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:46818
I20260812 01:09:01.028497 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22893
I20260812 01:09:01.038379 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 23095
I20260812 01:09:01.061160 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 23304
I20260812 01:09:01.077669 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22753
2026-08-12T01:09:01Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2396 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:01 dist-test-slave-tfpf krb5kdc[23527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:01 dist-test-slave-tfpf krb5kdc[23527](info): set up 2 sockets
Aug 12 01:09:01 dist-test-slave-tfpf krb5kdc[23527](info): commencing operation
krb5kdc: starting...
W20260812 01:09:02.447149 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:02 dist-test-slave-tfpf krb5kdc[23527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:02.601001 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:36225
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:36225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:02.692059 23545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:02.692375 23545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:02.692433 23545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:02.693722 23545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:02.693816 23545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:02.693845 23545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:02.693871 23545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:02.693904 23545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:02.695654 23545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:36225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:36225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:02.695996 23545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:02.696266 23545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:02.701864 23562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:02.701864 23557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:02.701906 23560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:02.702633 23545 server_base.cc:1034] running on GCE node
I20260812 01:09:02.702921 23545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:02.703341 23545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:02.704567 23545 hybrid_clock.cc:648] HybridClock initialized: now 1786496942704550 us; error 30 us; skew 500 ppm
Aug 12 01:09:02 dist-test-slave-tfpf krb5kdc[23527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:02.712903 23545 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:02.713439 23545 webserver.cc:460] Webserver started at http://127.17.130.62:45673/ using document root <none> and password file <none>
I20260812 01:09:02.713707 23545 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:02.713759 23545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:02.713898 23545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:02.715298 23545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "25eb00de03b2416cb94d489763be54c0"
format_stamp: "Formatted at 2026-08-12 01:09:02 on dist-test-slave-tfpf"
server_key: "da79d781d4d346dd75afc4980c612572"
server_key_iv: "1a0d22008c85c3129dd85c43b66cd6c3"
server_key_version: "encryptionkey@0"
I20260812 01:09:02.715715 23545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "25eb00de03b2416cb94d489763be54c0"
format_stamp: "Formatted at 2026-08-12 01:09:02 on dist-test-slave-tfpf"
server_key: "da79d781d4d346dd75afc4980c612572"
server_key_iv: "1a0d22008c85c3129dd85c43b66cd6c3"
server_key_version: "encryptionkey@0"
I20260812 01:09:02.717520 23545 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260812 01:09:02.718477 23571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:02.718847 23545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:02.718950 23545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "25eb00de03b2416cb94d489763be54c0"
format_stamp: "Formatted at 2026-08-12 01:09:02 on dist-test-slave-tfpf"
server_key: "da79d781d4d346dd75afc4980c612572"
server_key_iv: "1a0d22008c85c3129dd85c43b66cd6c3"
server_key_version: "encryptionkey@0"
I20260812 01:09:02.719058 23545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:02.753679 23545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:02.756268 23545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:02.756551 23545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:02.761476 23545 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:36225
I20260812 01:09:02.761539 23645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:36225 every 8 connection(s)
I20260812 01:09:02.762061 23545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:02.762902 23646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:02.765777 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0: Bootstrap starting.
I20260812 01:09:02.766462 23646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:02.766786 23646 log.cc:824] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:02.767593 23646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0: No bootstrap required, opened a new log
I20260812 01:09:02.769302 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 23545
I20260812 01:09:02.769423 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:02.769610 17928 external_mini_cluster.cc:1428] Setting key f053fdabfef96cf75f85eeb2264b0f58
I20260812 01:09:02.769542 23646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25eb00de03b2416cb94d489763be54c0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36225 } }
I20260812 01:09:02.769739 23646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:02.769824 23646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25eb00de03b2416cb94d489763be54c0, State: Initialized, Role: FOLLOWER
I20260812 01:09:02.769989 23646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25eb00de03b2416cb94d489763be54c0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36225 } }
I20260812 01:09:02.770099 23646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:02.770167 23646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:02.770232 23646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:02.771129 23646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25eb00de03b2416cb94d489763be54c0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36225 } }
I20260812 01:09:02.771273 23646 leader_election.cc:302] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25eb00de03b2416cb94d489763be54c0; no voters:
I20260812 01:09:02.771711 23646 leader_election.cc:288] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:02.771776 23653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:02.771912 23653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [term 1 LEADER]: Becoming Leader. State: Replica: 25eb00de03b2416cb94d489763be54c0, State: Running, Role: LEADER
I20260812 01:09:02.772029 23646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:02.772055 23653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25eb00de03b2416cb94d489763be54c0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36225 } }
I20260812 01:09:02.772547 23657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25eb00de03b2416cb94d489763be54c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25eb00de03b2416cb94d489763be54c0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36225 } } }
I20260812 01:09:02.772658 23657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:02.772958 23654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25eb00de03b2416cb94d489763be54c0. Latest consensus state: current_term: 1 leader_uuid: "25eb00de03b2416cb94d489763be54c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25eb00de03b2416cb94d489763be54c0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36225 } } }
I20260812 01:09:02.773337 23654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:02.773478 23666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:02.774212 23666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:02.775957 23666 catalog_manager.cc:1347] Generated new cluster ID: e72acf641fc9460ead4b5eedd3306674
I20260812 01:09:02.776013 23666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:09:02 dist-test-slave-tfpf krb5kdc[23527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:02.783247 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:02.770195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55564 (local address 127.17.130.62:36225)
0812 01:09:02.771856 (+ 1661us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:02.771866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:02.772095 (+ 229us) server_negotiation.cc:407] Connection header received
0812 01:09:02.772309 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:02.772317 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:02.772382 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:02.772469 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:02.773507 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:02.775255 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:02.776211 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:02.776650 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:02.776799 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:02.779420 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:02.779431 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:02.779432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:02.779452 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:02.781406 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:02.781847 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:02.781851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:02.781852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:02.781980 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:02.782220 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:02.782221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:02.782222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:02.782617 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:02.782824 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:02.782902 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 01:09:02.783054 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1610,"thread_start_us":1508,"threads_started":1}
I20260812 01:09:02.787657 23666 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:02.788300 23666 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:02.791232 23666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25eb00de03b2416cb94d489763be54c0: Generated new TSK 0
I20260812 01:09:02.791451 23666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:09:02.828022 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:36225
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:02.933741 23685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:02.933990 23685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:02.934032 23685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:02.935300 23685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:02.935364 23685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:02.935410 23685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:02.937397 23685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:02.937716 23685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:02.937985 23685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:02.944378 23698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:02.944603 23685 server_base.cc:1034] running on GCE node
W20260812 01:09:02.944404 23699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:02.944372 23702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:02.944947 23685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:02.945300 23685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:02.946509 23685 hybrid_clock.cc:648] HybridClock initialized: now 1786496942946481 us; error 45 us; skew 500 ppm
Aug 12 01:09:02 dist-test-slave-tfpf krb5kdc[23527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:02.949263 23685 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:09:02.949772 23685 webserver.cc:460] Webserver started at http://127.17.130.1:38943/ using document root <none> and password file <none>
I20260812 01:09:02.950060 23685 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:02.950111 23685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:02.950258 23685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:02.951355 23685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "94a484a6c3574b5aae1a0647fe8ab59b"
format_stamp: "Formatted at 2026-08-12 01:09:02 on dist-test-slave-tfpf"
server_key: "412a7200ac8139e8343a6ecbe1f84540"
server_key_iv: "372def6e5bd15958e644ef00067626f9"
server_key_version: "encryptionkey@0"
I20260812 01:09:02.951738 23685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "94a484a6c3574b5aae1a0647fe8ab59b"
format_stamp: "Formatted at 2026-08-12 01:09:02 on dist-test-slave-tfpf"
server_key: "412a7200ac8139e8343a6ecbe1f84540"
server_key_iv: "372def6e5bd15958e644ef00067626f9"
server_key_version: "encryptionkey@0"
I20260812 01:09:02.953284 23685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:02.954236 23711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:02.954469 23685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:02.954587 23685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "94a484a6c3574b5aae1a0647fe8ab59b"
format_stamp: "Formatted at 2026-08-12 01:09:02 on dist-test-slave-tfpf"
server_key: "412a7200ac8139e8343a6ecbe1f84540"
server_key_iv: "372def6e5bd15958e644ef00067626f9"
server_key_version: "encryptionkey@0"
I20260812 01:09:02.954648 23685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:02.974426 23685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:02.976640 23685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:02.976785 23685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:02.976979 23685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:02.977349 23685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:02.977418 23685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:02.977464 23685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:02.977504 23685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:02.984769 23685 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:40469
I20260812 01:09:02.984814 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:40469 every 8 connection(s)
I20260812 01:09:02.985275 23685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:02.985844 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 23685
I20260812 01:09:02.985958 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:02.986169 17928 external_mini_cluster.cc:1428] Setting key 6b00582a86ab13c21e1044e1cbd26f6a
Aug 12 01:09:02 dist-test-slave-tfpf krb5kdc[23527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496942, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
I20260812 01:09:02.998543 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:02.986303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:48229 (local address 127.17.130.62:36225)
0812 01:09:02.986379 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:02.986382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:02.986867 (+ 485us) server_negotiation.cc:407] Connection header received
0812 01:09:02.987886 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:02.987887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:02.987905 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:02.987978 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:02.988593 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:02.989262 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:02.991111 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:02.991325 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:02.991400 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:02.994841 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:02.994843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:02.994843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:02.994865 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:02.996563 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:02.997416 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:02.997418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:02.997418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:02.997487 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:02.997850 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:02.997851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:02.997852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:02.998001 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:02.998073 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:02.998382 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 01:09:02.998464 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:09:02.998751 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:02.986273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:36225 (local address 127.17.130.1:48229)
0812 01:09:02.986722 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:02.986762 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:02.987769 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:02.988037 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:02.988040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:02.988309 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:02.988503 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:02.988508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:02.989355 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:02.989357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:02.990989 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:02.990993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:02.991432 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:02.991434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:02.991600 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:02.992316 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:02.992325 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:02.994716 (+ 2391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:02.996690 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:02.996700 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:02.996703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:02.997304 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:02.997564 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:02.997565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:02.997565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:02.997756 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:02.998081 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:02.998087 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:02.998225 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 01:09:02.998341 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 01:09:02.998479 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":117,"threads_started":1}
I20260812 01:09:02.999387 23882 heartbeater.cc:347] Connected to a master server at 127.17.130.62:36225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
I20260812 01:09:02.999537 23882 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:02.999769 23882 heartbeater.cc:511] Master 127.17.130.62:36225 requested a full tablet report, sending...
I20260812 01:09:03.000455 23592 ts_manager.cc:194] Registered new tserver with Master: 94a484a6c3574b5aae1a0647fe8ab59b (127.17.130.1:40469)
I20260812 01:09:03.001305 23592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:48229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:09:03.018249 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:36225
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:03.119244 23888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:03.119506 23888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:03.119537 23888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:03.120779 23888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:03.120833 23888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:03.120864 23888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:03.122725 23888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:03.123049 23888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:03.123302 23888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:03.128979 23895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:03.128988 23894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:03.129009 23897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:03.129999 23888 server_base.cc:1034] running on GCE node
I20260812 01:09:03.130314 23888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:03.130631 23888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:03.131834 23888 hybrid_clock.cc:648] HybridClock initialized: now 1786496943131790 us; error 66 us; skew 500 ppm
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:03.134866 23888 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:09:03.135466 23888 webserver.cc:460] Webserver started at http://127.17.130.2:32789/ using document root <none> and password file <none>
I20260812 01:09:03.135748 23888 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:03.135821 23888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:03.135985 23888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:03.137254 23888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "8a0312a7d3a6471ab271d7e2cdc6b466"
format_stamp: "Formatted at 2026-08-12 01:09:03 on dist-test-slave-tfpf"
server_key: "dd994eef2e99a53c2301fac03dccc95b"
server_key_iv: "2786fdd95ff20bce34b099093037e3f8"
server_key_version: "encryptionkey@0"
I20260812 01:09:03.137661 23888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "8a0312a7d3a6471ab271d7e2cdc6b466"
format_stamp: "Formatted at 2026-08-12 01:09:03 on dist-test-slave-tfpf"
server_key: "dd994eef2e99a53c2301fac03dccc95b"
server_key_iv: "2786fdd95ff20bce34b099093037e3f8"
server_key_version: "encryptionkey@0"
I20260812 01:09:03.139232 23888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:09:03.140261 23907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:03.140564 23888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:03.140692 23888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "8a0312a7d3a6471ab271d7e2cdc6b466"
format_stamp: "Formatted at 2026-08-12 01:09:03 on dist-test-slave-tfpf"
server_key: "dd994eef2e99a53c2301fac03dccc95b"
server_key_iv: "2786fdd95ff20bce34b099093037e3f8"
server_key_version: "encryptionkey@0"
I20260812 01:09:03.140911 23888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:03.159885 23888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:03.162639 23888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:03.162837 23888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:03.163126 23888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:03.163475 23888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:03.163535 23888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:03.163580 23888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:03.163606 23888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:03.170451 23888 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:34511
I20260812 01:09:03.170502 24077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:34511 every 8 connection(s)
I20260812 01:09:03.170894 23888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:03.176041 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 23888
I20260812 01:09:03.176168 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:03.176383 17928 external_mini_cluster.cc:1428] Setting key f7b364c504b38f16092bd0ea17e6e371
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:03.184144 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:03.171971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:52641 (local address 127.17.130.62:36225)
0812 01:09:03.172083 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:03.172085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:03.172302 (+ 217us) server_negotiation.cc:407] Connection header received
0812 01:09:03.173520 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:03.173522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:03.173545 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:03.173617 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:03.174406 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.174915 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.176460 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.176758 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.176839 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:03.179647 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:03.179650 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:03.179650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:03.179662 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:03.181797 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.182675 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.182677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.182677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.182763 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.183209 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.183210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.183211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.183396 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:03.183500 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:03.183865 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 01:09:03.183990 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 01:09:03.184341 24083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:03.171889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:36225 (local address 127.17.130.2:52641)
0812 01:09:03.172120 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:03.172167 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:03.173379 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:03.173739 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:03.173742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:03.174056 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:03.174276 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:03.174281 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.175077 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.175078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:03.176353 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:03.176356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.176875 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.176877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:03.177025 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:03.177989 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:03.177999 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:03.179522 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:03.182279 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:03.182288 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:03.182291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:03.182570 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:03.182880 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:03.182882 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:03.182882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:03.183115 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:03.183530 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:03.183537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:03.183690 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 01:09:03.183810 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 01:09:03.183987 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":129,"threads_started":1}
I20260812 01:09:03.185174 24078 heartbeater.cc:347] Connected to a master server at 127.17.130.62:36225
I20260812 01:09:03.185309 24078 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:03.185560 24078 heartbeater.cc:511] Master 127.17.130.62:36225 requested a full tablet report, sending...
I20260812 01:09:03.186084 23592 ts_manager.cc:194] Registered new tserver with Master: 8a0312a7d3a6471ab271d7e2cdc6b466 (127.17.130.2:34511)
I20260812 01:09:03.186496 23592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:52641
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:09:03.209548 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:36225
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:03.309867 24089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:03.310111 24089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:03.310138 24089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:03.311280 24089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:03.311319 24089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:03.311345 24089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:03.312907 24089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:36225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:03.313213 24089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:03.313421 24089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:03.319020 24103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:03.319033 24099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:03.319149 24097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:03.319470 24089 server_base.cc:1034] running on GCE node
I20260812 01:09:03.319691 24089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:03.319967 24089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:03.321174 24089 hybrid_clock.cc:648] HybridClock initialized: now 1786496943321143 us; error 46 us; skew 500 ppm
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:03.324548 24089 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:09:03.325026 24089 webserver.cc:460] Webserver started at http://127.17.130.3:39287/ using document root <none> and password file <none>
I20260812 01:09:03.325337 24089 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:03.325415 24089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:03.325577 24089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:03.326719 24089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "2a50cf5404aa4a9e97f736d223a63382"
format_stamp: "Formatted at 2026-08-12 01:09:03 on dist-test-slave-tfpf"
server_key: "65d67a3bc784969794efed3bb4655860"
server_key_iv: "2198a626e79c42b058e46db75fe21a0e"
server_key_version: "encryptionkey@0"
I20260812 01:09:03.327198 24089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "2a50cf5404aa4a9e97f736d223a63382"
format_stamp: "Formatted at 2026-08-12 01:09:03 on dist-test-slave-tfpf"
server_key: "65d67a3bc784969794efed3bb4655860"
server_key_iv: "2198a626e79c42b058e46db75fe21a0e"
server_key_version: "encryptionkey@0"
I20260812 01:09:03.328814 24089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:03.329806 24114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:03.330168 24089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:03.330282 24089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "2a50cf5404aa4a9e97f736d223a63382"
format_stamp: "Formatted at 2026-08-12 01:09:03 on dist-test-slave-tfpf"
server_key: "65d67a3bc784969794efed3bb4655860"
server_key_iv: "2198a626e79c42b058e46db75fe21a0e"
server_key_version: "encryptionkey@0"
I20260812 01:09:03.330375 24089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:03.346541 24089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:03.349278 24089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:03.349491 24089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:03.349784 24089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:03.350140 24089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:03.350203 24089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:03.350246 24089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:03.350292 24089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:03.358012 24089 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:33011
I20260812 01:09:03.358054 24279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:33011 every 8 connection(s)
I20260812 01:09:03.358788 24089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:03.367262 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 24089
I20260812 01:09:03.367352 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:03.367604 17928 external_mini_cluster.cc:1428] Setting key 4ffc5011edaebcbdbec5c7119e4f724a
I20260812 01:09:03.370743 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:03.359771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:41063 (local address 127.17.130.62:36225)
0812 01:09:03.359879 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:03.359882 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:03.360065 (+ 183us) server_negotiation.cc:407] Connection header received
0812 01:09:03.361145 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:03.361147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:03.361162 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:03.361246 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:03.362464 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.362995 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.364186 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.364380 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.364452 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:03.367120 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:03.367122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:03.367122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:03.367132 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:03.369176 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.369653 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.369654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.369655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.369725 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.369994 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.369995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.369995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.370129 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:03.370208 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:03.370543 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 01:09:03.370637 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 01:09:03.370998 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:03.359680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:36225 (local address 127.17.130.3:41063)
0812 01:09:03.359867 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:03.359917 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:03.360956 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:03.361306 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:03.361309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:03.361926 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:03.362268 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:03.362281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.363149 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.363150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:03.364022 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:03.364024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.364500 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.364502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:03.364633 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:03.365416 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:03.365425 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:03.366988 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:03.369281 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:03.369290 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:03.369293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:03.369557 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:03.369796 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:03.369797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:03.369798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:03.369921 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:03.370210 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:03.370219 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:03.370395 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 01:09:03.370518 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 01:09:03.370715 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
I20260812 01:09:03.371524 24284 heartbeater.cc:347] Connected to a master server at 127.17.130.62:36225
I20260812 01:09:03.371636 24284 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:03.371803 24284 heartbeater.cc:511] Master 127.17.130.62:36225 requested a full tablet report, sending...
I20260812 01:09:03.372340 23592 ts_manager.cc:194] Registered new tserver with Master: 2a50cf5404aa4a9e97f736d223a63382 (127.17.130.3:33011)
I20260812 01:09:03.372784 23592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:41063
I20260812 01:09:03.380772 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:09:03.410835 24292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:03.401710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.17.130.1:40469)
0812 01:09:03.401969 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:03.401975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:03.401988 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:03.402020 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:03.402031 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:03.402053 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:03.402146 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:03.402846 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.403606 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.404493 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.404845 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.404906 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:03.407582 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:03.407593 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:03.407593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:03.407601 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:03.409537 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.409971 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.409975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.409975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.410059 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.410281 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.410282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.410282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.410430 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:03.410544 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:03.410624 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 01:09:03.410694 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":122,"threads_started":1}
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[23527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:03.420432 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:03.412430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.17.130.62:36225)
0812 01:09:03.412490 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:03.412492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:03.412502 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:03.412556 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:03.412557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:03.412566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:03.412618 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:03.413145 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.413682 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.414372 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:03.414564 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:03.414634 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:03.417013 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:03.417015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:03.417015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:03.417022 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:03.418772 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.419499 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.419500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.419501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.419580 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:03.419867 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:03.419868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:03.419868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:03.420063 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:03.420174 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:03.420284 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 01:09:03.420362 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260812 01:09:03.420835 23592 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55572
I20260812 01:09:03.421553 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 23685
I20260812 01:09:03.451387 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 23888
I20260812 01:09:03.469499 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 24089
I20260812 01:09:03.479027 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 23545
2026-08-12T01:09:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2402 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[24298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[24298](info): set up 2 sockets
Aug 12 01:09:03 dist-test-slave-tfpf krb5kdc[24298](info): commencing operation
krb5kdc: starting...
W20260812 01:09:04.832113 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:04 dist-test-slave-tfpf krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:04.984542 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46479
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:05.077154 24316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:05.077405 24316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:05.077432 24316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:05.078697 24316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:05.078751 24316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:05.078766 24316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:05.078780 24316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:05.078794 24316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:05.080602 24316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:05.081257 24316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:05.081568 24316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:05.087097 24323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:05.087249 24316 server_base.cc:1034] running on GCE node
W20260812 01:09:05.087352 24322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:05.087110 24327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:05.087670 24316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:05.087996 24316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:05.089215 24316 hybrid_clock.cc:648] HybridClock initialized: now 1786496945089180 us; error 54 us; skew 500 ppm
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:05.092294 24316 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:05.092644 24316 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:05.092700 24316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:05.092803 24316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:05.093994 24316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "d8eddf21cfed401fa4432ee915aa34e9"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "7e912a755aff055dc9d2c5e0f706e9ec"
server_key_iv: "95e0bf2bc18717e709ed5eb902a78275"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.094551 24316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "d8eddf21cfed401fa4432ee915aa34e9"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "7e912a755aff055dc9d2c5e0f706e9ec"
server_key_iv: "95e0bf2bc18717e709ed5eb902a78275"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.096320 24316 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 01:09:05.097422 24336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.097680 24316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:05.097776 24316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "d8eddf21cfed401fa4432ee915aa34e9"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "7e912a755aff055dc9d2c5e0f706e9ec"
server_key_iv: "95e0bf2bc18717e709ed5eb902a78275"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.097844 24316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:05.112829 24316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:05.115762 24316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:05.115927 24316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:05.121007 24316 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46479
I20260812 01:09:05.121022 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46479 every 8 connection(s)
I20260812 01:09:05.121424 24316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:05.122568 24425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:05.125288 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9: Bootstrap starting.
I20260812 01:09:05.125990 24425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:05.126343 24425 log.cc:824] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:05.127260 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9: No bootstrap required, opened a new log
I20260812 01:09:05.129045 24425 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eddf21cfed401fa4432ee915aa34e9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46479 } }
I20260812 01:09:05.129283 24425 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:05.129310 24425 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8eddf21cfed401fa4432ee915aa34e9, State: Initialized, Role: FOLLOWER
I20260812 01:09:05.129483 24425 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eddf21cfed401fa4432ee915aa34e9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46479 } }
I20260812 01:09:05.129571 24425 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:05.129597 24425 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:05.129648 24425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:05.130605 24425 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eddf21cfed401fa4432ee915aa34e9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46479 } }
I20260812 01:09:05.130704 24425 leader_election.cc:302] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8eddf21cfed401fa4432ee915aa34e9; no voters:
I20260812 01:09:05.130944 24425 leader_election.cc:288] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:05.131057 24431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:05.131258 24425 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:05.131366 24431 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [term 1 LEADER]: Becoming Leader. State: Replica: d8eddf21cfed401fa4432ee915aa34e9, State: Running, Role: LEADER
I20260812 01:09:05.131448 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 24316
I20260812 01:09:05.131572 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:05.131563 24431 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eddf21cfed401fa4432ee915aa34e9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46479 } }
I20260812 01:09:05.131778 17928 external_mini_cluster.cc:1428] Setting key 54bb005f70d52f77e3f8efcadd2cc3c6
I20260812 01:09:05.131966 24434 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8eddf21cfed401fa4432ee915aa34e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eddf21cfed401fa4432ee915aa34e9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46479 } } }
I20260812 01:09:05.131966 24436 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8eddf21cfed401fa4432ee915aa34e9. Latest consensus state: current_term: 1 leader_uuid: "d8eddf21cfed401fa4432ee915aa34e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8eddf21cfed401fa4432ee915aa34e9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46479 } } }
I20260812 01:09:05.132205 24434 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:05.132212 24436 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:05.132664 24452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:05.133705 24452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:05.135723 24452 catalog_manager.cc:1347] Generated new cluster ID: 568efcde0ba34edd9ab540e6d560aa2a
I20260812 01:09:05.135789 24452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:05.145228 24452 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:05.145334 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.132507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58542 (local address 127.17.130.62:46479)
0812 01:09:05.132697 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.132707 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.132962 (+ 255us) server_negotiation.cc:407] Connection header received
0812 01:09:05.133240 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.133250 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.133316 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.133451 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:05.134343 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.135830 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.136643 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.137153 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.137270 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.140825 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:05.140835 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:05.140835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:05.140849 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:05.142760 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.143619 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.143623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.143623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.143738 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.143980 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.143981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.143981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.144381 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:05.144623 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.144732 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.145033 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260812 01:09:05.145972 24452 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:05.154116 24452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9: Generated new TSK 0
I20260812 01:09:05.154361 24452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:09:05.200763 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46479
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:05.307961 24469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:05.308269 24469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:05.308302 24469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:05.309705 24469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:05.309767 24469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:05.309798 24469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:05.311501 24469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:05.311735 24469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:05.311936 24469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:05.317414 24478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:05.317428 24479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:05.317423 24481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:05.317847 24469 server_base.cc:1034] running on GCE node
I20260812 01:09:05.318316 24469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:05.318769 24469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:05.319940 24469 hybrid_clock.cc:648] HybridClock initialized: now 1786496945319914 us; error 50 us; skew 500 ppm
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:05.322883 24469 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:09:05.323362 24469 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:05.323446 24469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:05.323603 24469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:05.324743 24469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "9d733cb65fc543e59c57cb167b822d0b"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "e4ff5fddbb4c3ae0024722386d811467"
server_key_iv: "42f8189c8e7400cdc909357a7869c417"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.325309 24469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "9d733cb65fc543e59c57cb167b822d0b"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "e4ff5fddbb4c3ae0024722386d811467"
server_key_iv: "42f8189c8e7400cdc909357a7869c417"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.326858 24469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:05.327790 24488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.328114 24469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:05.328241 24469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "9d733cb65fc543e59c57cb167b822d0b"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "e4ff5fddbb4c3ae0024722386d811467"
server_key_iv: "42f8189c8e7400cdc909357a7869c417"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.328342 24469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:05.348145 24469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:05.351070 24469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:05.351285 24469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:05.351588 24469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:05.351967 24469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:05.352027 24469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.352066 24469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:05.352114 24469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.362587 24469 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:45397
I20260812 01:09:05.362624 24664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:45397 every 8 connection(s)
I20260812 01:09:05.363238 24469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:05.369568 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 24469
I20260812 01:09:05.369700 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:05.369927 17928 external_mini_cluster.cc:1428] Setting key ced575f7916610ca286d081247ab3e4d
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:05.377542 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.364788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:49983 (local address 127.17.130.62:46479)
0812 01:09:05.364888 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.364890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.365212 (+ 322us) server_negotiation.cc:407] Connection header received
0812 01:09:05.365909 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.365910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.365943 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.366022 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:05.367336 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.367889 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.369991 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.370413 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.370502 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.374106 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:05.374108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:05.374108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:05.374125 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:05.375778 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.376325 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.376331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.376331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.376391 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.376695 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.376696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.376696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.376834 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:05.376914 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.377324 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.377429 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:09:05.377768 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.364650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46479 (local address 127.17.130.1:49983)
0812 01:09:05.364922 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:05.365012 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:05.365608 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:05.366203 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:05.366209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:05.366550 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:05.367171 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.367178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.368071 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.368073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.369751 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.369760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.370521 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.370522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.370748 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.371657 (+ 909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:05.371670 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:05.373894 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:05.375894 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:05.375909 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:05.375912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:05.376224 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:05.376462 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:05.376463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:05.376463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:05.376599 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:05.376918 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:05.376927 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:05.377168 (+ 241us) client_negotiation.cc:769] Sending connection context
0812 01:09:05.377285 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:09:05.377495 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":156,"threads_started":1}
I20260812 01:09:05.378849 24666 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46479
I20260812 01:09:05.378991 24666 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:05.379199 24666 heartbeater.cc:511] Master 127.17.130.62:46479 requested a full tablet report, sending...
I20260812 01:09:05.380088 24367 ts_manager.cc:194] Registered new tserver with Master: 9d733cb65fc543e59c57cb167b822d0b (127.17.130.1:45397)
I20260812 01:09:05.381104 24367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:49983
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:09:05.405361 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46479
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:05.503532 24671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:05.503870 24671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:05.503907 24671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:05.505981 24671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:05.506067 24671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:05.506117 24671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:05.508015 24671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:05.508342 24671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:05.508646 24671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:05.514883 24679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:05.515144 24683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:05.514990 24680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:05.515195 24671 server_base.cc:1034] running on GCE node
I20260812 01:09:05.515542 24671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:05.515851 24671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:05.517104 24671 hybrid_clock.cc:648] HybridClock initialized: now 1786496945517041 us; error 77 us; skew 500 ppm
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:05.520335 24671 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:09:05.520794 24671 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:05.520869 24671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:05.520974 24671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:05.522149 24671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "9db11e8c0c8340008f2d8450b33be30d"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "b7fd40b529c5cdfc567f9deb99583112"
server_key_iv: "30c746a4da5c60cd9b82670f2a1a90d7"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.522475 24671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "9db11e8c0c8340008f2d8450b33be30d"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "b7fd40b529c5cdfc567f9deb99583112"
server_key_iv: "30c746a4da5c60cd9b82670f2a1a90d7"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.523823 24671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:05.524740 24691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.524969 24671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:05.525112 24671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "9db11e8c0c8340008f2d8450b33be30d"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "b7fd40b529c5cdfc567f9deb99583112"
server_key_iv: "30c746a4da5c60cd9b82670f2a1a90d7"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.525187 24671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:05.551203 24671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:05.553776 24671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:05.553905 24671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:05.554103 24671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:05.554456 24671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:05.554494 24671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.554517 24671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:05.554530 24671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.560992 24671 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:44931
I20260812 01:09:05.561241 24882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:44931 every 8 connection(s)
I20260812 01:09:05.561431 24671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:05.564152 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 24671
I20260812 01:09:05.564278 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:05.564697 17928 external_mini_cluster.cc:1428] Setting key 9dd76a9f03efe7d67c55b7c1b3721b38
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:05.574438 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.562358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:59205 (local address 127.17.130.62:46479)
0812 01:09:05.562425 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.562428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.562654 (+ 226us) server_negotiation.cc:407] Connection header received
0812 01:09:05.563209 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.563210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.563224 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.563297 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:05.564400 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.565043 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.566626 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.566830 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.566898 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.570018 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:05.570020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:05.570021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:05.570031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:05.572065 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.572787 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.572789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.572789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.572890 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.573349 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.573350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.573351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.573528 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:05.573613 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.574195 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.574319 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 01:09:05.575057 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.562223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46479 (local address 127.17.130.2:59205)
0812 01:09:05.562509 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:05.562549 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:05.563091 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:05.563377 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:05.563381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:05.563675 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:05.564228 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.564234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.565205 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.565207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.566510 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.566514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.566928 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.566929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.567088 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.567787 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:05.567800 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:05.569855 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:05.572196 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:05.572209 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:05.572213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:05.572631 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:05.573001 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:05.573002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:05.573003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:05.573256 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:05.573621 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:05.573628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:05.573821 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 01:09:05.574188 (+ 367us) client_negotiation.cc:240] Negotiation successful
0812 01:09:05.574676 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":92,"threads_started":1}
I20260812 01:09:05.575923 24883 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46479
I20260812 01:09:05.576081 24883 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:05.576231 24883 heartbeater.cc:511] Master 127.17.130.62:46479 requested a full tablet report, sending...
I20260812 01:09:05.576721 24367 ts_manager.cc:194] Registered new tserver with Master: 9db11e8c0c8340008f2d8450b33be30d (127.17.130.2:44931)
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:09:05.577319 24367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:59205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:09:05.600484 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46479
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:05.698194 24893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:05.698405 24893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:05.698431 24893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:05.699712 24893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:05.699770 24893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:05.699800 24893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:05.701601 24893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:05.701810 24893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:05.702059 24893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:05.707660 24905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:05.707674 24900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:05.707904 24893 server_base.cc:1034] running on GCE node
W20260812 01:09:05.708045 24903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:05.708420 24893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:05.708737 24893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:05.709928 24893 hybrid_clock.cc:648] HybridClock initialized: now 1786496945709906 us; error 40 us; skew 500 ppm
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:05.712801 24893 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:09:05.713438 24893 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:05.713515 24893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:05.713675 24893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:05.715029 24893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "96417e0ac247422f94470cedd1282b2f"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "a993cd94d6040985b48adebdba2fa53f"
server_key_iv: "593052c5cf9ed049661d6f1e673f47fb"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.715559 24893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "96417e0ac247422f94470cedd1282b2f"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "a993cd94d6040985b48adebdba2fa53f"
server_key_iv: "593052c5cf9ed049661d6f1e673f47fb"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.717172 24893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 01:09:05.718058 24911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.718310 24893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:05.718393 24893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "96417e0ac247422f94470cedd1282b2f"
format_stamp: "Formatted at 2026-08-12 01:09:05 on dist-test-slave-tfpf"
server_key: "a993cd94d6040985b48adebdba2fa53f"
server_key_iv: "593052c5cf9ed049661d6f1e673f47fb"
server_key_version: "encryptionkey@0"
I20260812 01:09:05.718493 24893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:05.741053 24893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:05.743527 24893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:05.743706 24893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:05.744017 24893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:05.744365 24893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:05.744402 24893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.744426 24893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:05.744438 24893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:05.750839 24893 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:33807
I20260812 01:09:05.750980 25076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:33807 every 8 connection(s)
I20260812 01:09:05.751222 24893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:05.757798 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 24893
Aug 12 01:09:05 dist-test-slave-tfpf krb5kdc[24298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496945, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:05.757927 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:05.758159 17928 external_mini_cluster.cc:1428] Setting key 83b9e7befc2e23af9ea0f49790058f15
I20260812 01:09:05.761906 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.752126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:49777 (local address 127.17.130.62:46479)
0812 01:09:05.752215 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.752217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.752328 (+ 111us) server_negotiation.cc:407] Connection header received
0812 01:09:05.752881 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.752882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.752893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.752955 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:05.754031 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.754542 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.755599 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.755794 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.755854 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.758526 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:05.758528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:05.758528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:05.758536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:05.760316 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.760807 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.760808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.760808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.760874 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.761195 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.761196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.761197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.761328 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:05.761409 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.761721 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.761810 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 01:09:05.762110 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.752019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46479 (local address 127.17.130.3:49777)
0812 01:09:05.752193 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:05.752247 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:05.752751 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:05.753133 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:05.753137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:05.753406 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:05.753929 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.753934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.754662 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.754663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.755510 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.755512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.755876 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.755877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.756183 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.756788 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:05.756796 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:05.758384 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:05.760452 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:05.760460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:05.760463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:05.760719 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:05.760951 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:05.760952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:05.760952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:05.761128 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:05.761400 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:05.761407 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:05.761570 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 01:09:05.761682 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 01:09:05.761857 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260812 01:09:05.762754 25077 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46479
I20260812 01:09:05.762862 25077 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:05.763067 25077 heartbeater.cc:511] Master 127.17.130.62:46479 requested a full tablet report, sending...
I20260812 01:09:05.763530 24367 ts_manager.cc:194] Registered new tserver with Master: 96417e0ac247422f94470cedd1282b2f (127.17.130.3:33807)
I20260812 01:09:05.763921 24367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:49777
I20260812 01:09:05.770663 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:09:05.778589 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.772101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58558 (local address 127.17.130.62:46479)
0812 01:09:05.772164 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.772166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.772316 (+ 150us) server_negotiation.cc:407] Connection header received
0812 01:09:05.772398 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.772399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.772410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.772463 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:05.773045 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.773593 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.774354 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.774552 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.774633 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.775901 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:05.775902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:05.775902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:05.775908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:05.777327 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.777705 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.777706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.777706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.777768 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:05.777976 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:05.777977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:05.777978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:05.778153 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:05.778265 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.778387 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.778469 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:09:05.780607 24367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:05.782461 24367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:05.789798 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.785689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47244 (local address 127.17.130.1:45397)
0812 01:09:05.785917 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.785921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.785939 (+ 18us) server_negotiation.cc:407] Connection header received
0812 01:09:05.786004 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.786006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.786023 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.786341 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:05.786423 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.787280 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.788558 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.789348 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.789420 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.789481 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.789669 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.789712 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":117,"threads_started":1}
I20260812 01:09:05.789788 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.785429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:45397 (local address 127.0.0.1:47244)
0812 01:09:05.785566 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:05.785607 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:05.785755 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:05.786160 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:05.786164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:05.786167 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:05.786250 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.786253 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.787389 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.787390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.788458 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.788461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.789434 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.789436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.789542 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.789557 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 01:09:05.789642 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:09:05.789698 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260812 01:09:05.790350 25101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.785767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:33807 (local address 127.0.0.1:42474)
0812 01:09:05.786174 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:05.786184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:05.786231 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:05.786646 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:05.786647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:05.786649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:05.786690 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.786692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.787955 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.787956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.788920 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.788923 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.789973 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.789974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.790036 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.790040 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:05.790221 (+ 181us) client_negotiation.cc:240] Negotiation successful
0812 01:09:05.790274 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":57,"thread_start_us":96,"threads_started":1}
I20260812 01:09:05.790380 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.785926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.17.130.3:33807)
0812 01:09:05.786117 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.786435 (+ 318us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.786445 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:05.786495 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.786497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.786511 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.786642 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:05.786744 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.787839 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.789134 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.789823 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.789972 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.790036 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.790244 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.790290 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":109,"threads_started":1}
I20260812 01:09:05.790737 24563 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7df54c8dcc4da0b9a376210a08d8cd (DEFAULT_TABLE table=test-table [id=2469a3822d7344ca893a585a5c3a41f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:05.790885 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.785971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:44931 (local address 127.0.0.1:58080)
0812 01:09:05.786323 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:05.786331 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:05.786382 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:05.786568 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:05.786569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:05.786572 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:05.786641 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.786643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.787961 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.787962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.789139 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:05.789143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.790671 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.790672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:05.790742 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.790747 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:05.790784 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 01:09:05.790822 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":61,"threads_started":1}
I20260812 01:09:05.791000 24976 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7df54c8dcc4da0b9a376210a08d8cd (DEFAULT_TABLE table=test-table [id=2469a3822d7344ca893a585a5c3a41f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:05.791206 24563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7df54c8dcc4da0b9a376210a08d8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:05.791291 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:05.786125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.17.130.2:44931)
0812 01:09:05.786327 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:05.786331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:05.786376 (+ 45us) server_negotiation.cc:407] Connection header received
0812 01:09:05.786449 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:05.786451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:05.786466 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:05.786549 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:05.786734 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.787839 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.789240 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:05.790351 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:05.791064 (+ 713us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:05.791128 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:05.791156 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 01:09:05.791201 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260812 01:09:05.791391 24976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7df54c8dcc4da0b9a376210a08d8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:05.791956 24770 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7df54c8dcc4da0b9a376210a08d8cd (DEFAULT_TABLE table=test-table [id=2469a3822d7344ca893a585a5c3a41f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:05.792318 24770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7df54c8dcc4da0b9a376210a08d8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:05.794137 25106 tablet_bootstrap.cc:492] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: Bootstrap starting.
I20260812 01:09:05.794919 25107 tablet_bootstrap.cc:492] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Bootstrap starting.
I20260812 01:09:05.794950 25106 tablet_bootstrap.cc:654] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:05.795523 25106 log.cc:824] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:05.795634 25107 tablet_bootstrap.cc:654] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:05.795876 25109 tablet_bootstrap.cc:492] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Bootstrap starting.
I20260812 01:09:05.796279 25107 log.cc:824] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:05.796439 25106 tablet_bootstrap.cc:492] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: No bootstrap required, opened a new log
I20260812 01:09:05.796533 25106 ts_tablet_manager.cc:1397] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:09:05.796567 25109 tablet_bootstrap.cc:654] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:05.797240 25107 tablet_bootstrap.cc:492] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: No bootstrap required, opened a new log
I20260812 01:09:05.797248 25109 log.cc:824] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:05.797384 25107 ts_tablet_manager.cc:1397] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:09:05.798149 25109 tablet_bootstrap.cc:492] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: No bootstrap required, opened a new log
I20260812 01:09:05.798223 25109 ts_tablet_manager.cc:1397] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:09:05.798590 25106 raft_consensus.cc:348] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:05.798731 25106 raft_consensus.cc:374] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:05.798754 25106 raft_consensus.cc:729] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d733cb65fc543e59c57cb167b822d0b, State: Initialized, Role: FOLLOWER
I20260812 01:09:05.798880 25106 consensus_queue.cc:260] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:05.799173 24666 heartbeater.cc:503] Master 127.17.130.62:46479 was elected leader, sending a full tablet report...
I20260812 01:09:05.799180 25106 ts_tablet_manager.cc:1428] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:05.800076 25107 raft_consensus.cc:348] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:05.800309 25107 raft_consensus.cc:374] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:05.800401 25107 raft_consensus.cc:729] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96417e0ac247422f94470cedd1282b2f, State: Initialized, Role: FOLLOWER
I20260812 01:09:05.800374 25109 raft_consensus.cc:348] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:05.800593 25109 raft_consensus.cc:374] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:05.800542 25107 consensus_queue.cc:260] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:05.800649 25109 raft_consensus.cc:729] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db11e8c0c8340008f2d8450b33be30d, State: Initialized, Role: FOLLOWER
I20260812 01:09:05.800787 25109 consensus_queue.cc:260] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:05.800891 25107 ts_tablet_manager.cc:1428] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:05.801151 25077 heartbeater.cc:503] Master 127.17.130.62:46479 was elected leader, sending a full tablet report...
I20260812 01:09:05.801420 25109 ts_tablet_manager.cc:1428] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:05.801555 24883 heartbeater.cc:503] Master 127.17.130.62:46479 was elected leader, sending a full tablet report...
W20260812 01:09:05.812670 24885 tablet.cc:2369] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:09:05.864625 24667 tablet.cc:2369] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:09:06.000892 25118 raft_consensus.cc:482] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:06.001297 25118 raft_consensus.cc:504] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
W20260812 01:09:06.001968 25078 tablet.cc:2369] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:09:06.001988 25118 leader_election.cc:288] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d733cb65fc543e59c57cb167b822d0b (127.17.130.1:45397), 9db11e8c0c8340008f2d8450b33be30d (127.17.130.2:44931)
I20260812 01:09:06.006326 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.002566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:37579 (local address 127.17.130.2:44931)
0812 01:09:06.002637 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.002639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.002652 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:06.002747 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.002747 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.002755 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.002832 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:06.002980 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.003658 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.005244 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.005835 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.005912 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:06.005923 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.006206 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.006251 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:09:06.006326 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.002427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:55661 (local address 127.17.130.1:45397)
0812 01:09:06.002482 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.002484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.002498 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:09:06.002559 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.002560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.002576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.002629 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:06.002819 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.003595 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.005244 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.005819 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.005915 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:06.005923 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.006206 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.006251 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:09:06.006337 25080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.002310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:45397 (local address 127.17.130.3:55661)
0812 01:09:06.002370 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.002384 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.002485 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.002642 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.002643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.002645 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:06.002730 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:06.002732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.003736 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.003737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:06.005103 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:06.005107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.005977 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.005978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:06.006078 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:06.006080 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.006161 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.006223 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":81}
I20260812 01:09:06.006345 25120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.002432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:44931 (local address 127.17.130.3:37579)
0812 01:09:06.002610 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.002620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.002662 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.002848 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.002849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.002851 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:06.002912 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:06.002913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.003741 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.003741 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:06.005103 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:06.005107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.005971 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.005972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:06.006074 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:06.006079 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.006161 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.006223 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":39,"thread_start_us":130,"threads_started":1}
I20260812 01:09:06.006844 24798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7df54c8dcc4da0b9a376210a08d8cd" candidate_uuid: "96417e0ac247422f94470cedd1282b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db11e8c0c8340008f2d8450b33be30d" is_pre_election: true
I20260812 01:09:06.006843 24592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7df54c8dcc4da0b9a376210a08d8cd" candidate_uuid: "96417e0ac247422f94470cedd1282b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d733cb65fc543e59c57cb167b822d0b" is_pre_election: true
I20260812 01:09:06.007021 24592 raft_consensus.cc:2454] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96417e0ac247422f94470cedd1282b2f in term 0.
I20260812 01:09:06.007022 24798 raft_consensus.cc:2454] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96417e0ac247422f94470cedd1282b2f in term 0.
I20260812 01:09:06.007326 24914 leader_election.cc:302] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96417e0ac247422f94470cedd1282b2f, 9db11e8c0c8340008f2d8450b33be30d; no voters:
I20260812 01:09:06.007476 25118 raft_consensus.cc:2781] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:06.007540 25118 raft_consensus.cc:482] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:06.007568 25118 raft_consensus.cc:3037] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.008438 25118 raft_consensus.cc:504] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:06.008600 25118 leader_election.cc:288] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [CANDIDATE]: Term 1 election: Requested vote from peers 9d733cb65fc543e59c57cb167b822d0b (127.17.130.1:45397), 9db11e8c0c8340008f2d8450b33be30d (127.17.130.2:44931)
I20260812 01:09:06.008773 24592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7df54c8dcc4da0b9a376210a08d8cd" candidate_uuid: "96417e0ac247422f94470cedd1282b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d733cb65fc543e59c57cb167b822d0b"
I20260812 01:09:06.008839 24798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7df54c8dcc4da0b9a376210a08d8cd" candidate_uuid: "96417e0ac247422f94470cedd1282b2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db11e8c0c8340008f2d8450b33be30d"
I20260812 01:09:06.008864 24592 raft_consensus.cc:3037] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.008944 24798 raft_consensus.cc:3037] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.009682 24592 raft_consensus.cc:2454] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96417e0ac247422f94470cedd1282b2f in term 1.
I20260812 01:09:06.009775 24798 raft_consensus.cc:2454] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96417e0ac247422f94470cedd1282b2f in term 1.
I20260812 01:09:06.009860 24914 leader_election.cc:302] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96417e0ac247422f94470cedd1282b2f, 9d733cb65fc543e59c57cb167b822d0b; no voters:
I20260812 01:09:06.009984 25118 raft_consensus.cc:2781] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:06.010201 25118 raft_consensus.cc:686] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 1 LEADER]: Becoming Leader. State: Replica: 96417e0ac247422f94470cedd1282b2f, State: Running, Role: LEADER
I20260812 01:09:06.010318 25118 consensus_queue.cc:237] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } }
I20260812 01:09:06.011355 24363 catalog_manager.cc:5495] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 96417e0ac247422f94470cedd1282b2f (127.17.130.3). New cstate: current_term: 1 leader_uuid: "96417e0ac247422f94470cedd1282b2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96417e0ac247422f94470cedd1282b2f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 } health_report { overall_health: UNKNOWN } } }
I20260812 01:09:06.023590 25103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.020851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42476 (local address 127.17.130.3:33807)
0812 01:09:06.020921 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.020923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.020955 (+ 32us) server_negotiation.cc:407] Connection header received
0812 01:09:06.021098 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.021099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.021127 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.021218 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:06.021378 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.021939 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.022725 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:06.022930 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:06.023007 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:06.023204 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:06.023318 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:06.023424 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.023443 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.023465 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:09:06.025923 24798 raft_consensus.cc:1261] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 1 FOLLOWER]: Refusing update from remote peer 96417e0ac247422f94470cedd1282b2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:06.025923 24592 raft_consensus.cc:1261] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 1 FOLLOWER]: Refusing update from remote peer 96417e0ac247422f94470cedd1282b2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:06.026315 25121 consensus_queue.cc:1035] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d733cb65fc543e59c57cb167b822d0b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:06.026460 25118 consensus_queue.cc:1035] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db11e8c0c8340008f2d8450b33be30d" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:06.029443 25128 mvcc.cc:205] Tried to move back new op lower bound from 7317491490918117376 to 7317491490858819584. Current Snapshot: MvccSnapshot[applied={T|T < 7317491490918117376}]
I20260812 01:09:06.032385 24367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:06.032526 24367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:06.033389 24367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8eddf21cfed401fa4432ee915aa34e9: Sending DeleteTablet for 3 replicas of tablet 9e7df54c8dcc4da0b9a376210a08d8cd
I20260812 01:09:06.033754 24563 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7df54c8dcc4da0b9a376210a08d8cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:06 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:47244
I20260812 01:09:06.033785 24976 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7df54c8dcc4da0b9a376210a08d8cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:06 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42474
I20260812 01:09:06.033874 24770 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7df54c8dcc4da0b9a376210a08d8cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:06 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:58080
I20260812 01:09:06.034092 25142 tablet_replica.cc:330] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b: stopping tablet replica
I20260812 01:09:06.034217 25142 raft_consensus.cc:2229] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:06.034323 25142 raft_consensus.cc:2258] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9d733cb65fc543e59c57cb167b822d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:06.034402 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 24469
I20260812 01:09:06.034513 25143 tablet_replica.cc:330] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: stopping tablet replica
I20260812 01:09:06.034601 25143 raft_consensus.cc:2229] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:06.034981 25143 raft_consensus.cc:2258] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:06.035429 25143 ts_tablet_manager.cc:1905] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:06.037294 25143 ts_tablet_manager.cc:1918] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:06.037362 25143 log.cc:1196] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/9e7df54c8dcc4da0b9a376210a08d8cd
I20260812 01:09:06.037609 25143 ts_tablet_manager.cc:1939] T 9e7df54c8dcc4da0b9a376210a08d8cd P 96417e0ac247422f94470cedd1282b2f: Deleting consensus metadata
I20260812 01:09:06.038069 24340 catalog_manager.cc:4841] TS 96417e0ac247422f94470cedd1282b2f (127.17.130.3:33807): tablet 9e7df54c8dcc4da0b9a376210a08d8cd (table test-table [id=2469a3822d7344ca893a585a5c3a41f7]) successfully deleted
I20260812 01:09:06.046221 25144 tablet_replica.cc:330] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: stopping tablet replica
I20260812 01:09:06.046411 25144 raft_consensus.cc:2229] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:06.046557 25144 raft_consensus.cc:2258] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:06.048792 25144 ts_tablet_manager.cc:1905] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:06.051251 25144 ts_tablet_manager.cc:1918] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:06.051373 25144 log.cc:1196] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/9e7df54c8dcc4da0b9a376210a08d8cd
I20260812 01:09:06.051720 25144 ts_tablet_manager.cc:1939] T 9e7df54c8dcc4da0b9a376210a08d8cd P 9db11e8c0c8340008f2d8450b33be30d: Deleting consensus metadata
I20260812 01:09:06.052309 24340 catalog_manager.cc:4841] TS 9db11e8c0c8340008f2d8450b33be30d (127.17.130.2:44931): tablet 9e7df54c8dcc4da0b9a376210a08d8cd (table test-table [id=2469a3822d7344ca893a585a5c3a41f7]) successfully deleted
W20260812 01:09:06.058176 24340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:45397 (error 108)
I20260812 01:09:06.058847 25104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.058688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:45397 (local address 127.0.0.1:47254)
0812 01:09:06.058758 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.058801 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:45397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:09:06.058950 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 24671
W20260812 01:09:06.069448 24340 catalog_manager.cc:4584] TS 9d733cb65fc543e59c57cb167b822d0b (127.17.130.1:45397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e7df54c8dcc4da0b9a376210a08d8cd: Network error: Client connection negotiation failed: client connection to 127.17.130.1:45397: connect: Connection refused (error 111)
I20260812 01:09:06.079005 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 24893
I20260812 01:09:06.099604 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 24316
2026-08-12T01:09:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2619 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T01:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:06Z Disabled control of system clock
I20260812 01:09:06.138661 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43467
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:43467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:09:06.233381 25152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:06.233618 25152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:06.233646 25152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:06.234848 25152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:06.234889 25152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:06.234901 25152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:06.234915 25152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:06.234926 25152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:06.236878 25152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:43467
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:06.237507 25152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:06.237844 25152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:06.244669 25164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.244721 25169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.244711 25167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:06.244776 25152 server_base.cc:1034] running on GCE node
I20260812 01:09:06.245337 25152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:06.245777 25152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:06.247006 25152 hybrid_clock.cc:648] HybridClock initialized: now 1786496946246983 us; error 49 us; skew 500 ppm
I20260812 01:09:06.248571 25152 webserver.cc:460] Webserver started at http://127.17.130.62:44383/ using document root <none> and password file <none>
I20260812 01:09:06.248896 25152 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:06.248981 25152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:06.249197 25152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:06.250522 25152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "25c68ed6e4c0490d8143de6f12f2663b"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "e69a72eb8a37ab199730e7c234a5850f"
server_key_iv: "2ee9c2e2afc4c37b26ba448cdc0080b9"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.250962 25152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "25c68ed6e4c0490d8143de6f12f2663b"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "e69a72eb8a37ab199730e7c234a5850f"
server_key_iv: "2ee9c2e2afc4c37b26ba448cdc0080b9"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.252617 25152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:06.253696 25181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.253954 25152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:06.254096 25152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "25c68ed6e4c0490d8143de6f12f2663b"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "e69a72eb8a37ab199730e7c234a5850f"
server_key_iv: "2ee9c2e2afc4c37b26ba448cdc0080b9"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.254184 25152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:06.257831 25152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:06.258123 25152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:06.258277 25152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:06.263034 25152 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:43467
I20260812 01:09:06.263090 25267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:43467 every 8 connection(s)
I20260812 01:09:06.263456 25152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:06.264252 25268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:06.265249 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 25152
I20260812 01:09:06.265381 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:06.265612 17928 external_mini_cluster.cc:1428] Setting key ccb058c1a01d8133bd1acde81e8faf25
I20260812 01:09:06.268664 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.266284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.17.130.62:43467)
0812 01:09:06.266517 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.266525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.266553 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:09:06.267057 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.267071 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.267117 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.267357 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.268109 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.268113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.268113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.268117 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.268160 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.268172 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.268237 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.268399 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.268485 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260812 01:09:06.269232 25268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b: Bootstrap starting.
I20260812 01:09:06.269899 25268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:06.270211 25268 log.cc:824] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:06.271006 25268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b: No bootstrap required, opened a new log
I20260812 01:09:06.272420 25268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c68ed6e4c0490d8143de6f12f2663b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43467 } }
I20260812 01:09:06.272590 25268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:06.272663 25268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25c68ed6e4c0490d8143de6f12f2663b, State: Initialized, Role: FOLLOWER
I20260812 01:09:06.272807 25268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c68ed6e4c0490d8143de6f12f2663b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43467 } }
I20260812 01:09:06.272904 25268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:06.273036 25268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:06.273152 25268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.274343 25268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c68ed6e4c0490d8143de6f12f2663b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43467 } }
I20260812 01:09:06.274484 25268 leader_election.cc:302] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25c68ed6e4c0490d8143de6f12f2663b; no voters:
I20260812 01:09:06.274677 25268 leader_election.cc:288] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:06.274772 25276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:06.275007 25276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [term 1 LEADER]: Becoming Leader. State: Replica: 25c68ed6e4c0490d8143de6f12f2663b, State: Running, Role: LEADER
I20260812 01:09:06.275074 25268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:06.275153 25276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c68ed6e4c0490d8143de6f12f2663b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43467 } }
I20260812 01:09:06.275564 25279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25c68ed6e4c0490d8143de6f12f2663b. Latest consensus state: current_term: 1 leader_uuid: "25c68ed6e4c0490d8143de6f12f2663b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c68ed6e4c0490d8143de6f12f2663b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43467 } } }
I20260812 01:09:06.275539 25278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25c68ed6e4c0490d8143de6f12f2663b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25c68ed6e4c0490d8143de6f12f2663b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43467 } } }
I20260812 01:09:06.275733 25279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:06.275796 25278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:06.276366 25298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:06.276975 25298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:06.278824 25298 catalog_manager.cc:1347] Generated new cluster ID: 13542e9ba6bf480c9885a86d86fbb8bf
I20260812 01:09:06.278884 25298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:06.301151 25298 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:06.301373 25298 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:06.314363 25298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b: Generated new TSK 0
I20260812 01:09:06.314630 25298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:09:06.324910 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43467
--builtin_ntp_servers=127.17.130.20:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:09:06.421182 25310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:06.421461 25310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:06.421504 25310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:06.422811 25310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:06.422904 25310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:06.422957 25310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:06.424449 25310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:06.424743 25310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:06.425119 25310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:06.430605 25317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.430601 25316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.430629 25322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:06.430960 25310 server_base.cc:1034] running on GCE node
I20260812 01:09:06.431247 25310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:06.431528 25310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:06.432704 25310 hybrid_clock.cc:648] HybridClock initialized: now 1786496946432694 us; error 34 us; skew 500 ppm
I20260812 01:09:06.434268 25310 webserver.cc:460] Webserver started at http://127.17.130.1:45771/ using document root <none> and password file <none>
I20260812 01:09:06.434536 25310 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:06.434612 25310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:06.434772 25310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:06.435900 25310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "3c33b65753764d11980a9f25f8bc6f12"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "38c39fa9242b4d1167c1fc3c6743a91c"
server_key_iv: "cb95d5f55dfd9800865bbc604649b86d"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.436283 25310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "3c33b65753764d11980a9f25f8bc6f12"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "38c39fa9242b4d1167c1fc3c6743a91c"
server_key_iv: "cb95d5f55dfd9800865bbc604649b86d"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.437847 25310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:06.438961 25328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.439232 25310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:06.439368 25310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "3c33b65753764d11980a9f25f8bc6f12"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "38c39fa9242b4d1167c1fc3c6743a91c"
server_key_iv: "cb95d5f55dfd9800865bbc604649b86d"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.439461 25310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:06.443271 25310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:06.443588 25310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:06.443758 25310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:06.444010 25310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:06.444465 25310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:06.444536 25310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.444573 25310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:06.444621 25310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.451071 25310 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:44043
I20260812 01:09:06.451249 25502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:44043 every 8 connection(s)
I20260812 01:09:06.451541 25310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:06.452050 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 25310
I20260812 01:09:06.452172 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:06.452396 17928 external_mini_cluster.cc:1428] Setting key 12e9b5830e01673b4debd6164d698336
I20260812 01:09:06.453810 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43467
--builtin_ntp_servers=127.17.130.20:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 01:09:06.455541 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.452463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:57425 (local address 127.17.130.62:43467)
0812 01:09:06.452527 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.452530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.452884 (+ 354us) server_negotiation.cc:407] Connection header received
0812 01:09:06.453497 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.453498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.453504 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.453554 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.455285 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.455286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.455287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.455291 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.455306 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.455311 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.455370 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.455470 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.455483 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:09:06.455888 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.452422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43467 (local address 127.17.130.1:57425)
0812 01:09:06.452670 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.452733 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.453380 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.453676 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.453680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.453691 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.455087 (+ 1396us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.455088 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.455129 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.455169 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.455390 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.455397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.455400 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.455480 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.455555 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260812 01:09:06.456485 25503 heartbeater.cc:347] Connected to a master server at 127.17.130.62:43467
I20260812 01:09:06.456629 25503 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:06.456813 25503 heartbeater.cc:511] Master 127.17.130.62:43467 requested a full tablet report, sending...
I20260812 01:09:06.457448 25208 ts_manager.cc:194] Registered new tserver with Master: 3c33b65753764d11980a9f25f8bc6f12 (127.17.130.1:44043)
W20260812 01:09:06.543167 25509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:06.543515 25509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:06.543674 25509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:06.544934 25509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:06.545013 25509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:06.545099 25509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:06.546559 25509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:06.546847 25509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:06.547181 25509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:06.553488 25521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.553550 25519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.553546 25523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:06.553725 25509 server_base.cc:1034] running on GCE node
I20260812 01:09:06.553998 25509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:06.554318 25509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:06.555532 25509 hybrid_clock.cc:648] HybridClock initialized: now 1786496946555467 us; error 74 us; skew 500 ppm
I20260812 01:09:06.557046 25509 webserver.cc:460] Webserver started at http://127.17.130.2:37313/ using document root <none> and password file <none>
I20260812 01:09:06.557391 25509 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:06.557493 25509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:06.557641 25509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:06.559396 25509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "25ece1feb931498bbe17c86de65f1229"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "a846becd3ef8a3a62e5e291382e70351"
server_key_iv: "15650b61cfc200557a66599389d20c6b"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.559953 25509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "25ece1feb931498bbe17c86de65f1229"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "a846becd3ef8a3a62e5e291382e70351"
server_key_iv: "15650b61cfc200557a66599389d20c6b"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.562052 25509 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260812 01:09:06.563241 25531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.563455 25509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:06.563586 25509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "25ece1feb931498bbe17c86de65f1229"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "a846becd3ef8a3a62e5e291382e70351"
server_key_iv: "15650b61cfc200557a66599389d20c6b"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.563689 25509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:06.567327 25509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:06.567620 25509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:06.567773 25509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:06.567996 25509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:06.568480 25509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:06.568542 25509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.568621 25509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:06.568651 25509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.575582 25509 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:43503
I20260812 01:09:06.575623 25690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:43503 every 8 connection(s)
I20260812 01:09:06.576042 25509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:06.579520 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.576822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:39421 (local address 127.17.130.62:43467)
0812 01:09:06.576875 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.576878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.577296 (+ 418us) server_negotiation.cc:407] Connection header received
0812 01:09:06.577881 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.577883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.577890 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.577981 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.579303 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.579305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.579305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.579310 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.579325 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.579329 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.579395 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.579439 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.579456 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:09:06.579742 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.576824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43467 (local address 127.17.130.2:39421)
0812 01:09:06.577170 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.577213 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.577757 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.578063 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.578068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.578079 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.579145 (+ 1066us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.579145 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.579176 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.579205 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.579390 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.579392 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.579394 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.579447 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.579496 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":105,"threads_started":1}
I20260812 01:09:06.580305 25691 heartbeater.cc:347] Connected to a master server at 127.17.130.62:43467
I20260812 01:09:06.580402 25691 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:06.580556 25691 heartbeater.cc:511] Master 127.17.130.62:43467 requested a full tablet report, sending...
I20260812 01:09:06.580951 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 25509
I20260812 01:09:06.581023 25208 ts_manager.cc:194] Registered new tserver with Master: 25ece1feb931498bbe17c86de65f1229 (127.17.130.2:43503)
I20260812 01:09:06.581060 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:06.581261 17928 external_mini_cluster.cc:1428] Setting key 826c94e714d2898c04740339a8cd297b
I20260812 01:09:06.582307 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43467
--builtin_ntp_servers=127.17.130.20:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 01:09:06.669468 25695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:06.669684 25695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:06.669708 25695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:06.670815 25695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:06.670881 25695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:06.670930 25695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:06.672920 25695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:06.673214 25695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:06.673518 25695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:06.679371 25702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.679371 25705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:06.679365 25703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:06.679689 25695 server_base.cc:1034] running on GCE node
I20260812 01:09:06.679908 25695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:06.680131 25695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:06.681281 25695 hybrid_clock.cc:648] HybridClock initialized: now 1786496946681251 us; error 60 us; skew 500 ppm
I20260812 01:09:06.683002 25695 webserver.cc:460] Webserver started at http://127.17.130.3:46491/ using document root <none> and password file <none>
I20260812 01:09:06.683269 25695 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:06.683318 25695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:06.683421 25695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:06.684549 25695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "d766c49d07d0466eb2e76f4c37f6eb12"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "5cdede3e7e04a2376b42d983602778ef"
server_key_iv: "a8c620c0b952ba3895f8e4c39e60505a"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.684880 25695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "d766c49d07d0466eb2e76f4c37f6eb12"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "5cdede3e7e04a2376b42d983602778ef"
server_key_iv: "a8c620c0b952ba3895f8e4c39e60505a"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.686574 25695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:06.687357 25716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.687557 25695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:06.687637 25695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "d766c49d07d0466eb2e76f4c37f6eb12"
format_stamp: "Formatted at 2026-08-12 01:09:06 on dist-test-slave-tfpf"
server_key: "5cdede3e7e04a2376b42d983602778ef"
server_key_iv: "a8c620c0b952ba3895f8e4c39e60505a"
server_key_version: "encryptionkey@0"
I20260812 01:09:06.687693 25695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:06.691107 25695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:06.691342 25695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:06.691437 25695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:06.691599 25695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:06.691849 25695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:06.691879 25695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.691900 25695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:06.691912 25695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:06.697810 25695 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:41899
I20260812 01:09:06.697855 25894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:41899 every 8 connection(s)
I20260812 01:09:06.698196 25695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:06.698614 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 25695
I20260812 01:09:06.698747 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:06.698973 17928 external_mini_cluster.cc:1428] Setting key 76f4f414542e881d4168f3a94a0d52c5
I20260812 01:09:06.701861 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.699090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:35077 (local address 127.17.130.62:43467)
0812 01:09:06.699168 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.699171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.699364 (+ 193us) server_negotiation.cc:407] Connection header received
0812 01:09:06.700069 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.700070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.700077 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.700125 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.701614 (+ 1489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.701616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.701616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.701621 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.701637 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.701641 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.701710 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.701792 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.701806 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:09:06.702039 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.699035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43467 (local address 127.17.130.3:35077)
0812 01:09:06.699228 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.699270 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.699883 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.700223 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.700227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.700239 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.701404 (+ 1165us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.701405 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.701436 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.701467 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.701712 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.701714 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.701717 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.701781 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.701831 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260812 01:09:06.702530 25896 heartbeater.cc:347] Connected to a master server at 127.17.130.62:43467
I20260812 01:09:06.702626 25896 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:06.702770 25896 heartbeater.cc:511] Master 127.17.130.62:43467 requested a full tablet report, sending...
I20260812 01:09:06.703203 25208 ts_manager.cc:194] Registered new tserver with Master: d766c49d07d0466eb2e76f4c37f6eb12 (127.17.130.3:41899)
I20260812 01:09:06.711413 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:09:06.714455 25274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.713179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51728 (local address 127.17.130.62:43467)
0812 01:09:06.713269 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.713271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.713371 (+ 100us) server_negotiation.cc:407] Connection header received
0812 01:09:06.713473 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.713474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.713481 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.713536 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.714199 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.714200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.714200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.714205 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.714219 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.714224 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.714296 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.714358 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.714374 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 01:09:06.716259 25208 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:06.718142 25208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:06.722496 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.720629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:43503 (local address 127.0.0.1:38134)
0812 01:09:06.720780 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.720823 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.720904 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.721374 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.721380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.721386 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.721950 (+ 564us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.721951 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.721987 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.722015 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.722302 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.722310 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.722314 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.722387 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.722403 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260812 01:09:06.722666 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.720630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:41899 (local address 127.0.0.1:60992)
0812 01:09:06.721001 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.721012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.721047 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.721790 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.721791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.721794 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.722274 (+ 480us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.722275 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.722283 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.722298 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.722515 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.722516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.722517 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.722575 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.722594 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":1,"thread_start_us":146,"threads_started":1}
I20260812 01:09:06.723097 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.720715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.17.130.2:43503)
0812 01:09:06.720935 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.720940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.720954 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:09:06.720986 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.720989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.721000 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.721169 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.722143 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.722151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.722151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.722157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.722196 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.722220 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.722289 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.722959 (+ 670us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.722977 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":139,"threads_started":1}
I20260812 01:09:06.723124 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.721216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.17.130.3:41899)
0812 01:09:06.721655 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.721658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.721667 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:06.721690 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.721691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.721698 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.721787 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.722366 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.722375 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.722375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.722380 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.722414 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.722439 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.722506 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.723012 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.723031 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":84,"threads_started":1}
I20260812 01:09:06.723775 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.720965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:44043 (local address 127.0.0.1:41802)
0812 01:09:06.721318 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.721329 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.721372 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.722578 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.722579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.722581 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.722953 (+ 372us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.722953 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.722960 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.722973 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.723633 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.723634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.723635 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.723701 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.723714 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":78,"threads_started":1}
I20260812 01:09:06.723824 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.721096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.17.130.1:44043)
0812 01:09:06.722186 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.722190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.722199 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:06.722225 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.722228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.722236 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.722309 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.723076 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.723085 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.723086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.723091 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.723129 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.723155 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.723622 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.723707 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.723725 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1049,"thread_start_us":92,"threads_started":1}
I20260812 01:09:06.723869 25789 tablet_service.cc:1467] Processing CreateTablet for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca (DEFAULT_TABLE table=test-table [id=2e150ef0c00d47d585fdc1f075dd08cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:06.723897 25605 tablet_service.cc:1467] Processing CreateTablet for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca (DEFAULT_TABLE table=test-table [id=2e150ef0c00d47d585fdc1f075dd08cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:06.724282 25789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:06.724351 25605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:06.724723 25398 tablet_service.cc:1467] Processing CreateTablet for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca (DEFAULT_TABLE table=test-table [id=2e150ef0c00d47d585fdc1f075dd08cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:06.725044 25398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:06.727546 25917 tablet_bootstrap.cc:492] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Bootstrap starting.
I20260812 01:09:06.728035 25919 tablet_bootstrap.cc:492] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Bootstrap starting.
I20260812 01:09:06.728528 25917 tablet_bootstrap.cc:654] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:06.728901 25919 tablet_bootstrap.cc:654] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:06.729050 25917 log.cc:824] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:06.729120 25920 tablet_bootstrap.cc:492] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12: Bootstrap starting.
I20260812 01:09:06.729396 25919 log.cc:824] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:06.729838 25920 tablet_bootstrap.cc:654] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:06.730026 25917 tablet_bootstrap.cc:492] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: No bootstrap required, opened a new log
I20260812 01:09:06.730105 25917 ts_tablet_manager.cc:1397] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:09:06.730208 25920 log.cc:824] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:06.730224 25919 tablet_bootstrap.cc:492] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: No bootstrap required, opened a new log
I20260812 01:09:06.730314 25919 ts_tablet_manager.cc:1397] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 01:09:06.731055 25920 tablet_bootstrap.cc:492] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12: No bootstrap required, opened a new log
I20260812 01:09:06.731130 25920 ts_tablet_manager.cc:1397] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:06.732519 25919 raft_consensus.cc:348] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.732690 25919 raft_consensus.cc:374] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:06.732735 25919 raft_consensus.cc:729] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d766c49d07d0466eb2e76f4c37f6eb12, State: Initialized, Role: FOLLOWER
I20260812 01:09:06.732854 25919 consensus_queue.cc:260] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.733172 25919 ts_tablet_manager.cc:1428] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:09:06.733101 25917 raft_consensus.cc:348] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.733284 25917 raft_consensus.cc:374] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:06.733312 25896 heartbeater.cc:503] Master 127.17.130.62:43467 was elected leader, sending a full tablet report...
I20260812 01:09:06.733326 25917 raft_consensus.cc:729] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ece1feb931498bbe17c86de65f1229, State: Initialized, Role: FOLLOWER
I20260812 01:09:06.733481 25917 consensus_queue.cc:260] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.733747 25917 ts_tablet_manager.cc:1428] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:09:06.733868 25691 heartbeater.cc:503] Master 127.17.130.62:43467 was elected leader, sending a full tablet report...
I20260812 01:09:06.733928 25920 raft_consensus.cc:348] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.734091 25920 raft_consensus.cc:374] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:06.734135 25920 raft_consensus.cc:729] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c33b65753764d11980a9f25f8bc6f12, State: Initialized, Role: FOLLOWER
I20260812 01:09:06.734262 25920 consensus_queue.cc:260] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.734539 25920 ts_tablet_manager.cc:1428] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260812 01:09:06.734699 25503 heartbeater.cc:503] Master 127.17.130.62:43467 was elected leader, sending a full tablet report...
W20260812 01:09:06.827337 25693 tablet.cc:2369] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:09:06.876463 25930 raft_consensus.cc:482] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:06.876619 25930 raft_consensus.cc:504] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.877147 25930 leader_election.cc:288] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d766c49d07d0466eb2e76f4c37f6eb12 (127.17.130.3:41899), 3c33b65753764d11980a9f25f8bc6f12 (127.17.130.1:44043)
I20260812 01:09:06.878793 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.877296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:41899 (local address 127.17.130.2:58491)
0812 01:09:06.877373 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.877387 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.877473 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.877660 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.877662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.877664 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.878191 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.878192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.878201 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.878217 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.878598 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.878599 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.878601 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.878687 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.878703 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:09:06.878830 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.877390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58491 (local address 127.17.130.3:41899)
0812 01:09:06.877468 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.877470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.877483 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:06.877589 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.877590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.877597 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.877651 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.878435 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.878436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.878436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.878442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.878481 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.878486 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.878564 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.878703 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.878734 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:09:06.878906 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.877425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:44043 (local address 127.17.130.2:53273)
0812 01:09:06.877627 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.877638 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:06.877699 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:06.877834 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:06.877835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:06.877837 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:06.878521 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:06.878522 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:06.878530 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:06.878545 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:06.878752 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:06.878753 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:06.878754 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 01:09:06.878807 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:09:06.878820 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260812 01:09:06.878907 25915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.877574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:53273 (local address 127.17.130.1:44043)
0812 01:09:06.877632 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.877634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.877711 (+ 77us) server_negotiation.cc:407] Connection header received
0812 01:09:06.877762 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.877763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.877768 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.877818 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.878643 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.878645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.878645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.878651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.878674 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.878679 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.878746 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.878808 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.878823 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:09:06.879153 25820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92ed7fe7aa040b2a9ef94d59ec1f6ca" candidate_uuid: "25ece1feb931498bbe17c86de65f1229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" is_pre_election: true
I20260812 01:09:06.879308 25820 raft_consensus.cc:2454] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25ece1feb931498bbe17c86de65f1229 in term 0.
I20260812 01:09:06.879241 25426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92ed7fe7aa040b2a9ef94d59ec1f6ca" candidate_uuid: "25ece1feb931498bbe17c86de65f1229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c33b65753764d11980a9f25f8bc6f12" is_pre_election: true
I20260812 01:09:06.879418 25426 raft_consensus.cc:2454] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 25ece1feb931498bbe17c86de65f1229 in term 0.
I20260812 01:09:06.879611 25537 leader_election.cc:302] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25ece1feb931498bbe17c86de65f1229, d766c49d07d0466eb2e76f4c37f6eb12; no voters:
I20260812 01:09:06.879761 25930 raft_consensus.cc:2781] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:06.879851 25930 raft_consensus.cc:482] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:06.879889 25930 raft_consensus.cc:3037] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.880659 25930 raft_consensus.cc:504] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.880811 25930 leader_election.cc:288] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [CANDIDATE]: Term 1 election: Requested vote from peers d766c49d07d0466eb2e76f4c37f6eb12 (127.17.130.3:41899), 3c33b65753764d11980a9f25f8bc6f12 (127.17.130.1:44043)
I20260812 01:09:06.881109 25426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92ed7fe7aa040b2a9ef94d59ec1f6ca" candidate_uuid: "25ece1feb931498bbe17c86de65f1229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c33b65753764d11980a9f25f8bc6f12"
I20260812 01:09:06.881114 25820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e92ed7fe7aa040b2a9ef94d59ec1f6ca" candidate_uuid: "25ece1feb931498bbe17c86de65f1229" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d766c49d07d0466eb2e76f4c37f6eb12"
I20260812 01:09:06.881210 25426 raft_consensus.cc:3037] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.881209 25820 raft_consensus.cc:3037] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:06.882220 25426 raft_consensus.cc:2454] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25ece1feb931498bbe17c86de65f1229 in term 1.
I20260812 01:09:06.882220 25820 raft_consensus.cc:2454] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 25ece1feb931498bbe17c86de65f1229 in term 1.
I20260812 01:09:06.882448 25537 leader_election.cc:302] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25ece1feb931498bbe17c86de65f1229, d766c49d07d0466eb2e76f4c37f6eb12; no voters:
I20260812 01:09:06.882565 25930 raft_consensus.cc:2781] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:06.882726 25930 raft_consensus.cc:686] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 1 LEADER]: Becoming Leader. State: Replica: 25ece1feb931498bbe17c86de65f1229, State: Running, Role: LEADER
I20260812 01:09:06.882835 25930 consensus_queue.cc:237] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } }
I20260812 01:09:06.883810 25208 catalog_manager.cc:5495] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 reported cstate change: term changed from 0 to 1, leader changed from <none> to 25ece1feb931498bbe17c86de65f1229 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "25ece1feb931498bbe17c86de65f1229" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "25ece1feb931498bbe17c86de65f1229" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 } health_report { overall_health: UNKNOWN } } }
I20260812 01:09:06.907567 25910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.906374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.17.130.2:43503)
0812 01:09:06.906453 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:06.906455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:06.906464 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:06.906513 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:06.906513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:06.906536 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:06.906608 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:06.907296 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:06.907297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:06.907298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:06.907302 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:06.907325 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:06.907329 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:06.907396 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:06.907449 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 01:09:06.907466 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:09:06.909647 25426 raft_consensus.cc:1261] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 3c33b65753764d11980a9f25f8bc6f12 [term 1 FOLLOWER]: Refusing update from remote peer 25ece1feb931498bbe17c86de65f1229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:06.909735 25820 raft_consensus.cc:1261] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 1 FOLLOWER]: Refusing update from remote peer 25ece1feb931498bbe17c86de65f1229: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:06.910143 25930 consensus_queue.cc:1035] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c33b65753764d11980a9f25f8bc6f12" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 44043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:06.910315 25934 consensus_queue.cc:1035] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d766c49d07d0466eb2e76f4c37f6eb12" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 41899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:06.914026 25941 mvcc.cc:205] Tried to move back new op lower bound from 7317491494538506240 to 7317491494432559104. Current Snapshot: MvccSnapshot[applied={T|T < 7317491494538506240}]
I20260812 01:09:06.914261 25938 mvcc.cc:205] Tried to move back new op lower bound from 7317491494538506240 to 7317491494432559104. Current Snapshot: MvccSnapshot[applied={T|T < 7317491494538506240 or (T in {7317491494538506240})}]
I20260812 01:09:06.915764 25942 mvcc.cc:205] Tried to move back new op lower bound from 7317491494538506240 to 7317491494432559104. Current Snapshot: MvccSnapshot[applied={T|T < 7317491494538506240 or (T in {7317491494538506240})}]
I20260812 01:09:06.915961 25208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:06.916070 25208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:06.917110 25208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25c68ed6e4c0490d8143de6f12f2663b: Sending DeleteTablet for 3 replicas of tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca
I20260812 01:09:06.917536 25605 tablet_service.cc:1514] Processing DeleteTablet for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:06 UTC) from {username='slave'} at 127.0.0.1:38134
I20260812 01:09:06.917562 25398 tablet_service.cc:1514] Processing DeleteTablet for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:06 UTC) from {username='slave'} at 127.0.0.1:41802
I20260812 01:09:06.917622 25789 tablet_service.cc:1514] Processing DeleteTablet for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:06 UTC) from {username='slave'} at 127.0.0.1:60992
I20260812 01:09:06.917905 25951 tablet_replica.cc:330] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: stopping tablet replica
I20260812 01:09:06.917951 25954 tablet_replica.cc:330] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: stopping tablet replica
I20260812 01:09:06.918061 25954 raft_consensus.cc:2229] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:06.918186 25954 raft_consensus.cc:2258] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:06.918195 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 25310
I20260812 01:09:06.919215 25954 ts_tablet_manager.cc:1905] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:06.921279 25954 ts_tablet_manager.cc:1918] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:06.921360 25954 log.cc:1196] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/e92ed7fe7aa040b2a9ef94d59ec1f6ca
I20260812 01:09:06.921677 25954 ts_tablet_manager.cc:1939] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P d766c49d07d0466eb2e76f4c37f6eb12: Deleting consensus metadata
I20260812 01:09:06.945384 25189 catalog_manager.cc:4841] TS d766c49d07d0466eb2e76f4c37f6eb12 (127.17.130.3:41899): tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca (table test-table [id=2e150ef0c00d47d585fdc1f075dd08cc]) successfully deleted
I20260812 01:09:06.945834 25951 raft_consensus.cc:2229] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:06.946018 25951 raft_consensus.cc:2258] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:06.946715 25951 ts_tablet_manager.cc:1905] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:06.949012 25951 ts_tablet_manager.cc:1918] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:06.949172 25951 log.cc:1196] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/e92ed7fe7aa040b2a9ef94d59ec1f6ca
I20260812 01:09:06.949506 25951 ts_tablet_manager.cc:1939] T e92ed7fe7aa040b2a9ef94d59ec1f6ca P 25ece1feb931498bbe17c86de65f1229: Deleting consensus metadata
I20260812 01:09:06.950053 25186 catalog_manager.cc:4841] TS 25ece1feb931498bbe17c86de65f1229 (127.17.130.2:43503): tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca (table test-table [id=2e150ef0c00d47d585fdc1f075dd08cc]) successfully deleted
W20260812 01:09:06.951896 25189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:44043 (error 108)
I20260812 01:09:06.952281 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 25509
I20260812 01:09:06.957139 25914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:06.952327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:44043 (local address 127.0.0.1:41816)
0812 01:09:06.956956 (+ 4629us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:06.957087 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:44043: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4591}
W20260812 01:09:06.965401 25189 catalog_manager.cc:4584] TS 3c33b65753764d11980a9f25f8bc6f12 (127.17.130.1:44043): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e92ed7fe7aa040b2a9ef94d59ec1f6ca: Network error: Client connection negotiation failed: client connection to 127.17.130.1:44043: connect: Connection refused (error 111)
I20260812 01:09:06.967862 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 25695
I20260812 01:09:06.978438 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 25152
2026-08-12T01:09:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (882 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 01:09:07.010944 17928 mini_oidc.cc:152] Starting JWKS server
I20260812 01:09:07.012253 17928 webserver.cc:460] Webserver started at https://127.0.0.1:34283/ using document root <none> and password file <none>
I20260812 01:09:07.012388 17928 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 01:09:07.012639 17928 webserver.cc:460] Webserver started at http://127.0.0.1:37993/ using document root <none> and password file <none>
2026-08-12T01:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:07Z Disabled control of system clock
I20260812 01:09:07.023761 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43083
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:40849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:43083
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34283/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/testchainca.cert with env {}
W20260812 01:09:07.114945 25964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:07.115180 25964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:07.115206 25964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:07.116340 25964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:07.116380 25964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:07.116393 25964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:07.116406 25964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:07.116420 25964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:09:07.116432 25964 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:09:07.116443 25964 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34283/jwks.json
I20260812 01:09:07.118090 25964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:43083
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34283/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:07.118315 25964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:07.118485 25964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:07.124018 25979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:07.123991 25977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:07.123991 25976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:07.124403 25964 server_base.cc:1034] running on GCE node
I20260812 01:09:07.124706 25964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:07.125036 25964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:07.126259 25964 hybrid_clock.cc:648] HybridClock initialized: now 1786496947126234 us; error 33 us; skew 500 ppm
I20260812 01:09:07.127785 25964 webserver.cc:460] Webserver started at http://127.17.130.62:40451/ using document root <none> and password file <none>
I20260812 01:09:07.128104 25964 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:07.128186 25964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:07.128348 25964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:07.129616 25964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "fac988e9d2794cd7ac0846d46297f331"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "0a54c2864670cc43bcded110d7751af1"
server_key_iv: "46aefff8699a0972155657418917fac4"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.130002 25964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "fac988e9d2794cd7ac0846d46297f331"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "0a54c2864670cc43bcded110d7751af1"
server_key_iv: "46aefff8699a0972155657418917fac4"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.131568 25964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:07.132519 25987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:07.132784 25964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:07.132887 25964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "fac988e9d2794cd7ac0846d46297f331"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "0a54c2864670cc43bcded110d7751af1"
server_key_iv: "46aefff8699a0972155657418917fac4"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.133090 25964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:07.148586 25964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:07.148978 25964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:07.149194 25964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:07.154001 25964 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:43083
I20260812 01:09:07.154037 26057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:43083 every 8 connection(s)
I20260812 01:09:07.154467 25964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:07.155539 26058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:07.158160 26058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331: Bootstrap starting.
I20260812 01:09:07.158819 26058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:07.159327 26058 log.cc:824] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:07.159811 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 25964
I20260812 01:09:07.159930 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:07.160177 17928 external_mini_cluster.cc:1428] Setting key 207ee8ac6c5ae66996f4fb3afd5f30db
I20260812 01:09:07.160311 26058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331: No bootstrap required, opened a new log
I20260812 01:09:07.163033 26058 raft_consensus.cc:348] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac988e9d2794cd7ac0846d46297f331" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43083 } }
I20260812 01:09:07.163215 26058 raft_consensus.cc:374] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:07.163267 26058 raft_consensus.cc:729] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac988e9d2794cd7ac0846d46297f331, State: Initialized, Role: FOLLOWER
I20260812 01:09:07.163407 26058 consensus_queue.cc:260] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac988e9d2794cd7ac0846d46297f331" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43083 } }
I20260812 01:09:07.163489 26058 raft_consensus.cc:388] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:07.163539 26058 raft_consensus.cc:482] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:07.163589 26058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:07.164346 26058 raft_consensus.cc:504] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac988e9d2794cd7ac0846d46297f331" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43083 } }
I20260812 01:09:07.164445 26058 leader_election.cc:302] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fac988e9d2794cd7ac0846d46297f331; no voters:
I20260812 01:09:07.164616 26058 leader_election.cc:288] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:07.164655 26066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:07.164804 26066 raft_consensus.cc:686] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [term 1 LEADER]: Becoming Leader. State: Replica: fac988e9d2794cd7ac0846d46297f331, State: Running, Role: LEADER
I20260812 01:09:07.164904 26066 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac988e9d2794cd7ac0846d46297f331" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43083 } }
I20260812 01:09:07.164947 26058 sys_catalog.cc:564] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:07.165268 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fac988e9d2794cd7ac0846d46297f331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac988e9d2794cd7ac0846d46297f331" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43083 } } }
I20260812 01:09:07.165354 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:07.165907 26069 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fac988e9d2794cd7ac0846d46297f331. Latest consensus state: current_term: 1 leader_uuid: "fac988e9d2794cd7ac0846d46297f331" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac988e9d2794cd7ac0846d46297f331" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43083 } } }
I20260812 01:09:07.165980 26069 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:07.166239 26072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:07.166874 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.160760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40262 (local address 127.17.130.62:43083)
0812 01:09:07.160992 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.160999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.161025 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:09:07.161736 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.161743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.161806 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.161921 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:07.162237 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.163123 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.164293 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.165013 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.165592 (+ 579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.166025 (+ 433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:07.166029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:07.166029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:07.166278 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:07.166324 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:07.166341 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:07.166473 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:07.166581 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:09:07.166672 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":104,"threads_started":1}
I20260812 01:09:07.167135 26072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:07.167937 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:09:07.168910 26072 catalog_manager.cc:1347] Generated new cluster ID: 0980629a80fe40468d810760685a697c
I20260812 01:09:07.168972 26072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:07.197925 26072 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:07.198834 26072 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:07.206427 26072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fac988e9d2794cd7ac0846d46297f331: Generated new TSK 0
I20260812 01:09:07.206657 26072 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 01:09:07.230494 26104 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 01:09:07.230715 26104 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 01:09:07.230921 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.214465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.17.130.62:43083)
0812 01:09:07.214515 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.214517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.214701 (+ 184us) server_negotiation.cc:407] Connection header received
0812 01:09:07.214770 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.214771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.214787 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.214871 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:09:07.215129 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.215763 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.217048 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.217355 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.217428 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.217786 (+ 358us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:09:07.230596 (+ 12810us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 01:09:07.230826 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40266: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:09:07.231001 26065 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40266: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 01:09:07.231443 26102 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:07.214323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43083 (local address 127.0.0.1:40266)
0812 01:09:07.214575 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:07.214588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:07.214672 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:07.214886 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:07.214887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:07.214889 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:09:07.214961 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.214963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.215917 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.215918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.216905 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.216908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.217446 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.217447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.217536 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.217562 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:09:07.230926 (+ 13364us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 01:09:07.231089 (+ 163us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.130.62:43083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":148,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.17.130.62:43083: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 01:09:07.232041 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 25964
2026-08-12T01:09:07Z chronyd exiting
I20260812 01:09:07.277441 17928 test_util.cc:175] -----------------------------------------------
I20260812 01:09:07.277530 17928 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786496918848844-17928-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (267 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 01:09:07.278393 17928 mini_oidc.cc:152] Starting JWKS server
I20260812 01:09:07.279070 17928 webserver.cc:460] Webserver started at https://127.0.0.1:39635/ using document root <none> and password file <none>
I20260812 01:09:07.279112 17928 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 01:09:07.279259 17928 webserver.cc:460] Webserver started at http://127.0.0.1:45065/ using document root <none> and password file <none>
2026-08-12T01:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:07Z Disabled control of system clock
I20260812 01:09:07.288733 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:45053
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:40997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:45053
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39635/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/testchainca.cert with env {}
W20260812 01:09:07.378525 26114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:07.378810 26114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:07.378855 26114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:07.380337 26114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:07.380426 26114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:07.380450 26114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:07.380474 26114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:07.380503 26114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:09:07.380529 26114 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:09:07.380566 26114 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39635/jwks.json
I20260812 01:09:07.382532 26114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:45053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:45053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39635/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:07.382899 26114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:07.383158 26114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:07.388876 26123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:07.388999 26114 server_base.cc:1034] running on GCE node
W20260812 01:09:07.388870 26125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:07.388891 26122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:07.389487 26114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:07.389830 26114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:07.391049 26114 hybrid_clock.cc:648] HybridClock initialized: now 1786496947391002 us; error 68 us; skew 500 ppm
I20260812 01:09:07.392567 26114 webserver.cc:460] Webserver started at http://127.17.130.62:43061/ using document root <none> and password file <none>
I20260812 01:09:07.392863 26114 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:07.393155 26114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:07.393342 26114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:07.394703 26114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "95411770879b438987879c732d56743b"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "49b645d990ac96fca193bda6427fa34d"
server_key_iv: "857a872f97ded34bda66da9ce2505b2c"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.395149 26114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "95411770879b438987879c732d56743b"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "49b645d990ac96fca193bda6427fa34d"
server_key_iv: "857a872f97ded34bda66da9ce2505b2c"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.396873 26114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:07.397859 26132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:07.398219 26114 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:07.398357 26114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "95411770879b438987879c732d56743b"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "49b645d990ac96fca193bda6427fa34d"
server_key_iv: "857a872f97ded34bda66da9ce2505b2c"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.398444 26114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:07.407837 26114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:07.408226 26114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:07.408396 26114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:07.413023 26114 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:45053
I20260812 01:09:07.413101 26217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:45053 every 8 connection(s)
I20260812 01:09:07.413525 26114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:07.414247 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 26114
I20260812 01:09:07.414374 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:07.414479 26218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:07.414616 17928 external_mini_cluster.cc:1428] Setting key 639c6ff3ba86bcd68bb9978c68558967
I20260812 01:09:07.418428 26218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b: Bootstrap starting.
I20260812 01:09:07.419149 26218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:07.419459 26218 log.cc:824] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:07.420472 26218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b: No bootstrap required, opened a new log
I20260812 01:09:07.421175 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.415308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.17.130.62:45053)
0812 01:09:07.415581 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.415589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.415615 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:09:07.416587 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.416596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.416669 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.416877 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:07.417136 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.418056 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.418957 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.419471 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.419632 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.420413 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:07.420417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:07.420417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:07.420635 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:07.420683 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:07.420694 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:07.420748 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:07.420864 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 01:09:07.420931 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260812 01:09:07.422472 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:09:07.423349 26218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95411770879b438987879c732d56743b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45053 } }
I20260812 01:09:07.423527 26218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:07.423573 26218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95411770879b438987879c732d56743b, State: Initialized, Role: FOLLOWER
I20260812 01:09:07.423695 26218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95411770879b438987879c732d56743b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45053 } }
I20260812 01:09:07.423784 26218 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:07.423818 26218 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:07.423890 26218 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:07.424456 26218 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95411770879b438987879c732d56743b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45053 } }
I20260812 01:09:07.424547 26218 leader_election.cc:302] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95411770879b438987879c732d56743b; no voters:
I20260812 01:09:07.424685 26218 leader_election.cc:288] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:07.424772 26225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:07.424953 26225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [term 1 LEADER]: Becoming Leader. State: Replica: 95411770879b438987879c732d56743b, State: Running, Role: LEADER
I20260812 01:09:07.424983 26218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:07.425097 26225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95411770879b438987879c732d56743b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45053 } }
I20260812 01:09:07.425479 26227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95411770879b438987879c732d56743b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95411770879b438987879c732d56743b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45053 } } }
I20260812 01:09:07.425503 26228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95411770879b438987879c732d56743b. Latest consensus state: current_term: 1 leader_uuid: "95411770879b438987879c732d56743b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95411770879b438987879c732d56743b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45053 } } }
I20260812 01:09:07.425592 26228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:07.425580 26227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:07.425835 26242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:07.426424 26242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:07.428426 26242 catalog_manager.cc:1347] Generated new cluster ID: e3750ee9d8a34c7889a9e8a0609b69a1
I20260812 01:09:07.428507 26242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:07.439512 26242 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:07.440122 26242 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:07.444883 26242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95411770879b438987879c732d56743b: Generated new TSK 0
I20260812 01:09:07.445150 26242 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 01:09:07.476564 26266 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 01:09:07.476691 26266 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 01:09:07.477190 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.454002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.17.130.62:45053)
0812 01:09:07.454054 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.454062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.454107 (+ 45us) server_negotiation.cc:407] Connection header received
0812 01:09:07.454208 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.454209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.454220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.454285 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:09:07.454434 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.454944 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.456128 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.456338 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.456405 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.456654 (+ 249us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:09:07.476866 (+ 20212us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 01:09:07.477121 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 01:09:07.477265 26222 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41858: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 01:09:07.477344 26264 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:07.453866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:45053 (local address 127.0.0.1:41858)
0812 01:09:07.454040 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:07.454052 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:07.454127 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:07.454300 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:07.454301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:07.454303 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:09:07.454369 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.454371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.455049 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.455050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.455985 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.455989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.456427 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.456428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.456501 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.456508 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:09:07.477096 (+ 20588us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 01:09:07.477235 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.130.62:45053: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39635/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260812 01:09:07.477818 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 26114
2026-08-12T01:09:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (249 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 01:09:07.527900 17928 mini_oidc.cc:152] Starting JWKS server
I20260812 01:09:07.528661 17928 webserver.cc:460] Webserver started at https://127.0.0.1:44963/ using document root <none> and password file <none>
I20260812 01:09:07.528733 17928 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 01:09:07.528905 17928 webserver.cc:460] Webserver started at http://127.0.0.1:43851/ using document root <none> and password file <none>
2026-08-12T01:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:07Z Disabled control of system clock
I20260812 01:09:07.538810 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44737
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:43431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44737
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44963/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 01:09:07.632726 26274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:07.632977 26274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:07.633001 26274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:07.634305 26274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:07.634353 26274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:07.634366 26274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:07.634380 26274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:07.634392 26274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:09:07.634447 26274 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:09:07.634464 26274 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44963/jwks.json
I20260812 01:09:07.635905 26274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44963/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:07.636209 26274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:07.636543 26274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:07.641904 26286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:07.641860 26282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:07.642082 26281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:07.642333 26274 server_base.cc:1034] running on GCE node
I20260812 01:09:07.642683 26274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:07.643033 26274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:07.644239 26274 hybrid_clock.cc:648] HybridClock initialized: now 1786496947644202 us; error 62 us; skew 500 ppm
I20260812 01:09:07.645835 26274 webserver.cc:460] Webserver started at http://127.17.130.62:35087/ using document root <none> and password file <none>
I20260812 01:09:07.646268 26274 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:07.646394 26274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:07.646708 26274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:07.648257 26274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "9bd4a17a6b4741fe8561b71b882234e4"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "c5aee7a18beeba37f38623410c00cebb"
server_key_iv: "ae96d5933ea164a6f76f3a883ce87f1c"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.648703 26274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "9bd4a17a6b4741fe8561b71b882234e4"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "c5aee7a18beeba37f38623410c00cebb"
server_key_iv: "ae96d5933ea164a6f76f3a883ce87f1c"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.650293 26274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 01:09:07.651259 26294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:07.651479 26274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:07.651604 26274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "9bd4a17a6b4741fe8561b71b882234e4"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "c5aee7a18beeba37f38623410c00cebb"
server_key_iv: "ae96d5933ea164a6f76f3a883ce87f1c"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.651684 26274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:07.673686 26274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:07.674093 26274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:07.674261 26274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:07.679139 26274 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44737
I20260812 01:09:07.679236 26383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44737 every 8 connection(s)
I20260812 01:09:07.679607 26274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:07.680418 26384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:07.683462 26384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4: Bootstrap starting.
I20260812 01:09:07.684093 26384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:07.684345 26384 log.cc:824] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:07.685032 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 26274
I20260812 01:09:07.685168 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:07.685207 26384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4: No bootstrap required, opened a new log
I20260812 01:09:07.685380 17928 external_mini_cluster.cc:1428] Setting key ef84cd8ba1c4901dd9ac096b262ae491
I20260812 01:09:07.687260 26384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd4a17a6b4741fe8561b71b882234e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44737 } }
I20260812 01:09:07.687409 26384 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:07.687433 26384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bd4a17a6b4741fe8561b71b882234e4, State: Initialized, Role: FOLLOWER
I20260812 01:09:07.687579 26384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd4a17a6b4741fe8561b71b882234e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44737 } }
I20260812 01:09:07.687683 26384 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:07.687729 26384 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:07.687778 26384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:07.688371 26384 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd4a17a6b4741fe8561b71b882234e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44737 } }
I20260812 01:09:07.688496 26384 leader_election.cc:302] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bd4a17a6b4741fe8561b71b882234e4; no voters:
I20260812 01:09:07.688710 26384 leader_election.cc:288] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:07.688737 26394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:07.688851 26394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [term 1 LEADER]: Becoming Leader. State: Replica: 9bd4a17a6b4741fe8561b71b882234e4, State: Running, Role: LEADER
I20260812 01:09:07.688941 26394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd4a17a6b4741fe8561b71b882234e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44737 } }
I20260812 01:09:07.689035 26384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:07.689309 26399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bd4a17a6b4741fe8561b71b882234e4. Latest consensus state: current_term: 1 leader_uuid: "9bd4a17a6b4741fe8561b71b882234e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd4a17a6b4741fe8561b71b882234e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44737 } } }
I20260812 01:09:07.689329 26397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bd4a17a6b4741fe8561b71b882234e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bd4a17a6b4741fe8561b71b882234e4" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44737 } } }
I20260812 01:09:07.689489 26397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:07.689471 26399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4 [sys.catalog]: This master's current role is: LEADER
W20260812 01:09:07.691201 26423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:09:07.691269 26423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:09:07.691334 26424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:07.691489 26424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:07.693259 26424 catalog_manager.cc:1347] Generated new cluster ID: c123746b3f34418da7e03ceb6398f2ff
I20260812 01:09:07.693279 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.685998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.17.130.62:44737)
0812 01:09:07.686253 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.686262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.686288 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:09:07.687270 (+ 982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.687279 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.687375 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.687569 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:07.687870 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.689489 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.690603 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.691271 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.691398 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.692421 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:07.692425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:07.692425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:07.692672 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:07.692719 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:07.692737 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:07.692817 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:07.692916 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 01:09:07.692998 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":163,"thread_start_us":113,"threads_started":1}
I20260812 01:09:07.693332 26424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:07.694411 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:09:07.708266 26424 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:07.708949 26424 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:07.716536 26424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bd4a17a6b4741fe8561b71b882234e4: Generated new TSK 0
I20260812 01:09:07.716769 26424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:09:07.730666 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.725973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.17.130.62:44737)
0812 01:09:07.726049 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.726051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.726232 (+ 181us) server_negotiation.cc:407] Connection header received
0812 01:09:07.726302 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.726303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.726318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.726414 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:09:07.726615 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.727413 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.728589 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.728965 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.729052 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.729319 (+ 267us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:09:07.730415 (+ 1096us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 01:09:07.730575 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260812 01:09:07.730747 26391 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 01:09:07.732566 26439 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:07.725944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44737 (local address 127.0.0.1:51148)
0812 01:09:07.726143 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:07.726155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:07.726233 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:07.726455 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:07.726456 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:07.726459 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:09:07.726520 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.726523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.727819 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.727821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.728478 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.728480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.729113 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.729114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.729177 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.729183 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:09:07.730546 (+ 1363us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 01:09:07.732458 (+ 1912us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.130.62:44737: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44963/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":1667,"thread_start_us":145,"threads_started":1}
I20260812 01:09:07.732904 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 26274
E20260812 01:09:07.746361 26440 webserver.cc:533] Webserver: sslize failed:
E20260812 01:09:07.746460 26440 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T01:09:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (250 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T01:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:07Z Disabled control of system clock
I20260812 01:09:07.787892 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46765
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:34263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46765
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/testchainca.cert with env {}
W20260812 01:09:07.880555 26450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:07.880803 26450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:07.880829 26450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:07.882112 26450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:07.882155 26450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:07.882169 26450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:07.882182 26450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:07.882195 26450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 01:09:07.882207 26450 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 01:09:07.882220 26450 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 01:09:07.883926 26450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:07.884172 26450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:07.884356 26450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:07.889727 26455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:07.889922 26450 server_base.cc:1034] running on GCE node
W20260812 01:09:07.889715 26462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:07.889714 26456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:07.890398 26450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:07.890781 26450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:07.892009 26450 hybrid_clock.cc:648] HybridClock initialized: now 1786496947891985 us; error 40 us; skew 500 ppm
I20260812 01:09:07.898572 26450 webserver.cc:460] Webserver started at http://127.17.130.62:36739/ using document root <none> and password file <none>
I20260812 01:09:07.899029 26450 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:07.899117 26450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:07.899286 26450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:07.900476 26450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "67cb706f8fd447498391078de7c87fbb"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "8a5420a295aa521ad3f629fe99deb705"
server_key_iv: "c2911977af96ff205812ca2a56524e40"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.900897 26450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "67cb706f8fd447498391078de7c87fbb"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "8a5420a295aa521ad3f629fe99deb705"
server_key_iv: "c2911977af96ff205812ca2a56524e40"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.902580 26450 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:07.903882 26481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:07.904349 26450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:09:07.904563 26450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "67cb706f8fd447498391078de7c87fbb"
format_stamp: "Formatted at 2026-08-12 01:09:07 on dist-test-slave-tfpf"
server_key: "8a5420a295aa521ad3f629fe99deb705"
server_key_iv: "c2911977af96ff205812ca2a56524e40"
server_key_version: "encryptionkey@0"
I20260812 01:09:07.904706 26450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:07.924418 26450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:07.924822 26450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:07.924993 26450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:07.930441 26450 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46765
I20260812 01:09:07.930524 26561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46765 every 8 connection(s)
I20260812 01:09:07.930963 26450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:07.932156 26562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:07.933892 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 26450
I20260812 01:09:07.934139 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:07.934355 17928 external_mini_cluster.cc:1428] Setting key a07e0a88bf807830f9dc03d4b3f49d2f
I20260812 01:09:07.935274 26562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb: Bootstrap starting.
I20260812 01:09:07.935957 26562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:07.936380 26562 log.cc:824] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:07.937239 26562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb: No bootstrap required, opened a new log
I20260812 01:09:07.939777 26562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cb706f8fd447498391078de7c87fbb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46765 } }
I20260812 01:09:07.939980 26562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:07.940034 26562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67cb706f8fd447498391078de7c87fbb, State: Initialized, Role: FOLLOWER
I20260812 01:09:07.940183 26562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cb706f8fd447498391078de7c87fbb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46765 } }
I20260812 01:09:07.940301 26562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:07.940351 26562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:07.940399 26562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:07.940829 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.935169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58928 (local address 127.17.130.62:46765)
0812 01:09:07.935371 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.935399 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.935432 (+ 33us) server_negotiation.cc:407] Connection header received
0812 01:09:07.935912 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.935918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.935981 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.936144 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:07.936406 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.937422 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.938533 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.939090 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.939189 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.940054 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:07.940060 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:07.940060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:07.940272 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:07.940307 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:07.940341 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:07.940411 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:07.940547 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 01:09:07.940638 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260812 01:09:07.941285 26562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cb706f8fd447498391078de7c87fbb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46765 } }
I20260812 01:09:07.941403 26562 leader_election.cc:302] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67cb706f8fd447498391078de7c87fbb; no voters:
I20260812 01:09:07.941628 26562 leader_election.cc:288] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:07.941658 26570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:07.941772 26570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [term 1 LEADER]: Becoming Leader. State: Replica: 67cb706f8fd447498391078de7c87fbb, State: Running, Role: LEADER
I20260812 01:09:07.941848 26570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cb706f8fd447498391078de7c87fbb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46765 } }
I20260812 01:09:07.941910 26562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:07.941962 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:09:07.942169 26570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67cb706f8fd447498391078de7c87fbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cb706f8fd447498391078de7c87fbb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46765 } } }
I20260812 01:09:07.942245 26570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:07.942379 26570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67cb706f8fd447498391078de7c87fbb. Latest consensus state: current_term: 1 leader_uuid: "67cb706f8fd447498391078de7c87fbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67cb706f8fd447498391078de7c87fbb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46765 } } }
I20260812 01:09:07.942453 26570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb [sys.catalog]: This master's current role is: LEADER
W20260812 01:09:07.943920 26590 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:09:07.944002 26590 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:09:07.944051 26575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:07.944193 26575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:07.945967 26575 catalog_manager.cc:1347] Generated new cluster ID: 9629af327e55457aa5664050e303799b
I20260812 01:09:07.946098 26575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:07.953274 26575 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:07.954041 26575 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:07.969975 26575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67cb706f8fd447498391078de7c87fbb: Generated new TSK 0
I20260812 01:09:07.970223 26575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:09:08.486817 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:07.977165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.17.130.62:46765)
0812 01:09:07.977213 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:07.977215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:07.977344 (+ 129us) server_negotiation.cc:407] Connection header received
0812 01:09:07.977409 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:07.977410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:07.977422 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:07.977493 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0812 01:09:07.977695 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.978311 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.979529 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.979736 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.979863 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.980098 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 01:09:08.486540 (+506442us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 01:09:08.486752 (+ 212us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58944: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 01:09:08.486905 26566 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58944: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 01:09:08.486997 26600 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:07.977018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46765 (local address 127.0.0.1:58944)
0812 01:09:07.977238 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:07.977251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:07.977333 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:07.977525 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:07.977526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:07.977528 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 01:09:07.977594 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.977596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.978412 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.978413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.979426 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:07.979429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:07.979893 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:07.979894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:07.979983 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:07.979990 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 01:09:08.486719 (+506729us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 01:09:08.486876 (+ 157us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.17.130.62:46765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":129,"threads_started":1}
I20260812 01:09:08.487344 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 26450
2026-08-12T01:09:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (734 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (111 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (114 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:08 dist-test-slave-tfpf krb5kdc[26620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:08 dist-test-slave-tfpf krb5kdc[26620](info): set up 2 sockets
Aug 12 01:09:08 dist-test-slave-tfpf krb5kdc[26620](info): commencing operation
krb5kdc: starting...
W20260812 01:09:10.079599 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:10.220949 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46107
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46107
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:10.318537 26649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:10.318830 26649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:10.318873 26649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:10.320472 26649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:10.320598 26649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:10.320633 26649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:10.320660 26649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:10.320693 26649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:10.322755 26649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46107
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:10.323221 26649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:10.323508 26649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:10.329126 26656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:10.329281 26649 server_base.cc:1034] running on GCE node
W20260812 01:09:10.329149 26658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.329126 26668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:10.329689 26649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:10.330040 26649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:10.331295 26649 hybrid_clock.cc:648] HybridClock initialized: now 1786496950331283 us; error 58 us; skew 500 ppm
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:10.334352 26649 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:10.334882 26649 webserver.cc:460] Webserver started at http://127.17.130.62:37315/ using document root <none> and password file <none>
I20260812 01:09:10.335212 26649 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:10.335290 26649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:10.335454 26649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:10.337322 26649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "bb1bffe99f364172be8e1b80479a47d3"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "fb64b127b10375bb8c9a6b2711279588"
server_key_iv: "0efffd83229ebf688c7ca3ca3a67dd78"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.337900 26649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "bb1bffe99f364172be8e1b80479a47d3"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "fb64b127b10375bb8c9a6b2711279588"
server_key_iv: "0efffd83229ebf688c7ca3ca3a67dd78"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.339566 26649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:09:10.340579 26678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.340880 26649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:10.341048 26649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "bb1bffe99f364172be8e1b80479a47d3"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "fb64b127b10375bb8c9a6b2711279588"
server_key_iv: "0efffd83229ebf688c7ca3ca3a67dd78"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.341190 26649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:10.348482 26649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:10.351111 26649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:10.351303 26649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:10.356392 26649 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46107
I20260812 01:09:10.356424 26757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46107 every 8 connection(s)
I20260812 01:09:10.356858 26649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:10.358247 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 26649
I20260812 01:09:10.358299 26759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:10.358472 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:10.358706 17928 external_mini_cluster.cc:1428] Setting key d14e9b0d9b295f91a6b0410d3b0dbfa2
I20260812 01:09:10.362375 26759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3: Bootstrap starting.
I20260812 01:09:10.363093 26759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:10.363464 26759 log.cc:824] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:10.364315 26759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3: No bootstrap required, opened a new log
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:10.366452 26759 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1bffe99f364172be8e1b80479a47d3" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46107 } }
I20260812 01:09:10.366631 26759 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:10.366688 26759 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1bffe99f364172be8e1b80479a47d3, State: Initialized, Role: FOLLOWER
I20260812 01:09:10.366839 26759 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1bffe99f364172be8e1b80479a47d3" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46107 } }
I20260812 01:09:10.366955 26759 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:10.367005 26759 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:10.367057 26759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:10.367858 26759 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1bffe99f364172be8e1b80479a47d3" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46107 } }
I20260812 01:09:10.367991 26759 leader_election.cc:302] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb1bffe99f364172be8e1b80479a47d3; no voters:
I20260812 01:09:10.368227 26759 leader_election.cc:288] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:10.368278 26766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:10.368448 26766 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [term 1 LEADER]: Becoming Leader. State: Replica: bb1bffe99f364172be8e1b80479a47d3, State: Running, Role: LEADER
I20260812 01:09:10.368563 26759 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:10.368593 26766 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1bffe99f364172be8e1b80479a47d3" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46107 } }
I20260812 01:09:10.369134 26768 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb1bffe99f364172be8e1b80479a47d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1bffe99f364172be8e1b80479a47d3" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46107 } } }
I20260812 01:09:10.369249 26768 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:10.369395 26768 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb1bffe99f364172be8e1b80479a47d3. Latest consensus state: current_term: 1 leader_uuid: "bb1bffe99f364172be8e1b80479a47d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb1bffe99f364172be8e1b80479a47d3" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46107 } } }
I20260812 01:09:10.369457 26768 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:10.369972 26780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:10.370664 26780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:10.371218 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.359350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.17.130.62:46107)
0812 01:09:10.359523 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:10.359531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:10.359558 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:09:10.360049 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:10.360056 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:10.360105 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:10.360234 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:10.361212 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.362582 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.363423 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.363993 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.364122 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.366816 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:10.366821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:10.366821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:10.366834 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:10.369107 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.369619 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.369622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.369622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.369746 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.370011 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.370012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.370012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.370407 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:10.370625 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:10.370760 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 01:09:10.371050 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260812 01:09:10.372874 26780 catalog_manager.cc:1347] Generated new cluster ID: 36457a5a89d444dd943fde6d1c23b597
I20260812 01:09:10.372931 26780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:10.383596 26780 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:10.384286 26780 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:10.390525 26780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb1bffe99f364172be8e1b80479a47d3: Generated new TSK 0
I20260812 01:09:10.390763 26780 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:09:10.429661 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46107
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:10.527683 26794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:10.527910 26794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:10.527940 26794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:10.529201 26794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:10.529244 26794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:10.529273 26794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:10.530848 26794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:10.531067 26794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:10.531275 26794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:10.537637 26806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.537701 26801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.537657 26808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:10.538406 26794 server_base.cc:1034] running on GCE node
I20260812 01:09:10.538852 26794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:10.539444 26794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:10.540715 26794 hybrid_clock.cc:648] HybridClock initialized: now 1786496950540674 us; error 60 us; skew 500 ppm
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:10.544369 26794 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:09:10.545042 26794 webserver.cc:460] Webserver started at http://127.17.130.1:33547/ using document root <none> and password file <none>
I20260812 01:09:10.545379 26794 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:10.545470 26794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:10.545634 26794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:10.546825 26794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "4624421c3ef549b9b5ba1460a208472d"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "f3da7a5487342810e1afd2032a96c4fe"
server_key_iv: "d08ecdd618e79df159610bc7967011a3"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.547247 26794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "4624421c3ef549b9b5ba1460a208472d"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "f3da7a5487342810e1afd2032a96c4fe"
server_key_iv: "d08ecdd618e79df159610bc7967011a3"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.549136 26794 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 01:09:10.550423 26819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.550693 26794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:10.550859 26794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "4624421c3ef549b9b5ba1460a208472d"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "f3da7a5487342810e1afd2032a96c4fe"
server_key_iv: "d08ecdd618e79df159610bc7967011a3"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.550953 26794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:10.582769 26794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:10.586238 26794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:10.586458 26794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:10.586773 26794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:10.587158 26794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:10.587231 26794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.587280 26794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:10.587311 26794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.595633 26794 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:46245
I20260812 01:09:10.595679 26999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:46245 every 8 connection(s)
I20260812 01:09:10.596077 26794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:10.598191 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 26794
I20260812 01:09:10.598337 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:10.598539 17928 external_mini_cluster.cc:1428] Setting key d9f0507ead1e023acb85f82900bceed4
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
I20260812 01:09:10.609946 27006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.596912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46107 (local address 127.17.130.1:51953)
0812 01:09:10.597206 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:10.597264 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:10.598556 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:10.598889 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:10.598894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:10.599322 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:10.599595 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:10.599601 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.600824 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.600825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:10.601830 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:10.601832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.602433 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.602434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:10.602556 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.603192 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:10.603202 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:10.605173 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:10.607803 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:10.607818 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:10.607822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:10.608244 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:10.608576 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:10.608578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:10.608579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:10.608772 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:10.609162 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:10.609171 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:10.609389 (+ 218us) client_negotiation.cc:769] Sending connection context
0812 01:09:10.609526 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 01:09:10.609739 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":143,"threads_started":1}
I20260812 01:09:10.609995 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.597028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:51953 (local address 127.17.130.62:46107)
0812 01:09:10.597166 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:10.597170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:10.597359 (+ 189us) server_negotiation.cc:407] Connection header received
0812 01:09:10.598729 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:10.598731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:10.598744 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:10.598803 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:10.599719 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.600610 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.601962 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.602303 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.602389 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.605410 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:10.605413 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:10.605414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:10.605428 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:10.607674 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.608372 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.608374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.608374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.608447 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.608877 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.608878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.608878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.609026 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:10.609184 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:10.609649 (+ 465us) server_negotiation.cc:299] Negotiation successful
0812 01:09:10.609886 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 01:09:10.611006 27000 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46107
I20260812 01:09:10.611157 27000 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:10.611346 27000 heartbeater.cc:511] Master 127.17.130.62:46107 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
I20260812 01:09:10.612535 26703 ts_manager.cc:194] Registered new tserver with Master: 4624421c3ef549b9b5ba1460a208472d (127.17.130.1:46245)
I20260812 01:09:10.613397 26703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:51953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:09:10.637854 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46107
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:10.735737 27010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:10.736135 27010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:10.736202 27010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:10.737711 27010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:10.737769 27010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:10.737803 27010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:10.739674 27010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:10.739984 27010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:10.740288 27010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:10.746562 27023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.746690 27021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.746775 27017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:10.746775 27010 server_base.cc:1034] running on GCE node
I20260812 01:09:10.747201 27010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:10.747498 27010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:10.748725 27010 hybrid_clock.cc:648] HybridClock initialized: now 1786496950748702 us; error 67 us; skew 500 ppm
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:10.752313 27010 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:09:10.752812 27010 webserver.cc:460] Webserver started at http://127.17.130.2:36807/ using document root <none> and password file <none>
I20260812 01:09:10.753180 27010 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:10.753248 27010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:10.753405 27010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:10.754689 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "53cc45fc0ccf47ce89e3b6b9b5de00e2"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "413fdabedf56a762380f04957904b7fe"
server_key_iv: "1d06ba0c4e93285f0ec704292bd792af"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.755151 27010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "53cc45fc0ccf47ce89e3b6b9b5de00e2"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "413fdabedf56a762380f04957904b7fe"
server_key_iv: "1d06ba0c4e93285f0ec704292bd792af"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.757007 27010 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 01:09:10.758191 27032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.758581 27010 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:10.758718 27010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "53cc45fc0ccf47ce89e3b6b9b5de00e2"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "413fdabedf56a762380f04957904b7fe"
server_key_iv: "1d06ba0c4e93285f0ec704292bd792af"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.758811 27010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:10.771894 27010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:10.775419 27010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:10.775637 27010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:10.775949 27010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:10.776324 27010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:10.776386 27010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.776434 27010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:10.776464 27010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.784085 27010 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:34563
I20260812 01:09:10.784184 27201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:34563 every 8 connection(s)
I20260812 01:09:10.784556 27010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:10.785802 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 27010
I20260812 01:09:10.785918 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:10.786145 17928 external_mini_cluster.cc:1428] Setting key 6b15f094f57c8d4812252ebf532e9dd4
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:09:10.797577 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.785565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:47953 (local address 127.17.130.62:46107)
0812 01:09:10.785653 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:10.785655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:10.785952 (+ 297us) server_negotiation.cc:407] Connection header received
0812 01:09:10.787146 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:10.787148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:10.787162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:10.787235 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:10.787904 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.788660 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.790290 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.790488 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.790610 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.793444 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:10.793446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:10.793446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:10.793457 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:10.795727 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.796279 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.796280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.796281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.796354 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.796670 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.796682 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.796683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.796859 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:10.796950 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:10.797347 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 01:09:10.797452 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:09:10.797799 27207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.785456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46107 (local address 127.17.130.2:47953)
0812 01:09:10.785641 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:10.785697 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:10.787027 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:10.787312 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:10.787315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:10.787599 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:10.787805 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:10.787808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.788812 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.788815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:10.790185 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:10.790189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.790624 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.790625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:10.790783 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.791415 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:10.791424 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:10.793326 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:10.795903 (+ 2577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:10.795913 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:10.795916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:10.796184 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:10.796464 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:10.796465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:10.796465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:10.796589 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:10.796958 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:10.796965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:10.797197 (+ 232us) client_negotiation.cc:769] Sending connection context
0812 01:09:10.797297 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 01:09:10.797489 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260812 01:09:10.798503 27204 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46107
I20260812 01:09:10.798642 27204 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:10.798867 27204 heartbeater.cc:511] Master 127.17.130.62:46107 requested a full tablet report, sending...
I20260812 01:09:10.799711 26703 ts_manager.cc:194] Registered new tserver with Master: 53cc45fc0ccf47ce89e3b6b9b5de00e2 (127.17.130.2:34563)
I20260812 01:09:10.800316 26703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:47953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:09:10.821489 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46107
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:10.921484 27210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:10.921756 27210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:10.921798 27210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:10.923065 27210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:10.923152 27210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:10.923202 27210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:10.926028 27210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:10.926368 27210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:10.926929 27210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:10.932855 27219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.932850 27222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:10.932850 27218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:10.933184 27210 server_base.cc:1034] running on GCE node
I20260812 01:09:10.933430 27210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:10.933722 27210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:10.934924 27210 hybrid_clock.cc:648] HybridClock initialized: now 1786496950934896 us; error 44 us; skew 500 ppm
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:10.937980 27210 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:09:10.938591 27210 webserver.cc:460] Webserver started at http://127.17.130.3:45061/ using document root <none> and password file <none>
I20260812 01:09:10.938884 27210 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:10.938937 27210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:10.939082 27210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:10.940300 27210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "49b81e1b64804b7fb5116dd3ff4698f6"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "a0edf327d6a5d4325de13ea6bba6ab78"
server_key_iv: "be03bde95269074e0fed3b703c99b5f3"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.940770 27210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "49b81e1b64804b7fb5116dd3ff4698f6"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "a0edf327d6a5d4325de13ea6bba6ab78"
server_key_iv: "be03bde95269074e0fed3b703c99b5f3"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.942601 27210 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 01:09:10.943763 27234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.944100 27210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:10.944233 27210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "49b81e1b64804b7fb5116dd3ff4698f6"
format_stamp: "Formatted at 2026-08-12 01:09:10 on dist-test-slave-tfpf"
server_key: "a0edf327d6a5d4325de13ea6bba6ab78"
server_key_iv: "be03bde95269074e0fed3b703c99b5f3"
server_key_version: "encryptionkey@0"
I20260812 01:09:10.944324 27210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:10.953747 27210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:10.956734 27210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:10.956947 27210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:10.957298 27210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:10.957679 27210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:10.957741 27210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.957787 27210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:10.957861 27210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:10.965490 27210 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:35899
I20260812 01:09:10.965584 27422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:35899 every 8 connection(s)
I20260812 01:09:10.966091 27210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:10.969211 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 27210
I20260812 01:09:10.969493 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:10.969748 17928 external_mini_cluster.cc:1428] Setting key 8ac7d90dfc8ffe1877cb148c918c8152
Aug 12 01:09:10 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496950, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:10.982537 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.970858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:59971 (local address 127.17.130.62:46107)
0812 01:09:10.971002 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:10.971004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:10.971013 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:10.971043 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:10.971044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:10.971055 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:10.971136 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:10.972137 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.973003 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.974901 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.975288 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.975401 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.978355 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:10.978356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:10.978357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:10.978367 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:10.980053 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.980823 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.980826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.980826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.980963 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:10.981548 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:10.981550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:10.981550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:10.981701 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:10.981816 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:10.982307 (+ 491us) server_negotiation.cc:299] Negotiation successful
0812 01:09:10.982423 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 01:09:10.982609 27426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:10.967161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46107 (local address 127.17.130.3:59971)
0812 01:09:10.967468 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:10.967536 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:10.969011 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:10.971244 (+ 2233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:10.971250 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:10.971726 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:10.972034 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:10.972040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.973378 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.973380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:10.974783 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:10.974787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:10.975423 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:10.975424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:10.975564 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:10.976155 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:10.976165 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:10.978163 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:10.980235 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:10.980244 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:10.980246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:10.980528 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:10.981244 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:10.981245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:10.981246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:10.981398 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:10.981804 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:10.981811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:10.981997 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 01:09:10.982124 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 01:09:10.982299 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":185,"threads_started":1}
I20260812 01:09:10.983232 27424 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46107
I20260812 01:09:10.983394 27424 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:10.983582 27424 heartbeater.cc:511] Master 127.17.130.62:46107 requested a full tablet report, sending...
I20260812 01:09:10.984113 26702 ts_manager.cc:194] Registered new tserver with Master: 49b81e1b64804b7fb5116dd3ff4698f6 (127.17.130.3:35899)
I20260812 01:09:10.984568 26702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:59971
I20260812 01:09:10.994539 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/test-admin.keytab.
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.016278 17928 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 01:09:11.018775 27431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:11.017429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.17.130.1:46245)
0812 01:09:11.017687 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:11.017698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:11.017711 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:11.017747 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:11.017749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:11.017765 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:11.017844 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:11.018658 (+ 814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34058: BlockingRecv error: recv got EOF from 127.0.0.1:34058 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":126,"threads_started":1}
W20260812 01:09:11.018965 17928 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc' of type FILE
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.020540 17928 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.021940 17928 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:09:11.031899 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:11.022636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.17.130.1:46245)
0812 01:09:11.022718 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:11.022720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:11.022730 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:11.022764 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:11.022765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:11.022776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:11.022836 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:11.023415 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:11.024380 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:11.025299 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:11.025674 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:11.025750 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:11.028570 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:11.028585 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:11.028585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:11.028593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:11.030528 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:11.030973 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:11.030983 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:11.030983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:11.031046 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:11.031266 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:11.031267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:11.031268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:11.031433 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:11.031639 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:11.031688 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 01:09:11.031774 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 01:09:11.034060 27431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:11.033374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.17.130.1:46245)
0812 01:09:11.033434 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:11.033437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:11.033538 (+ 101us) server_negotiation.cc:407] Connection header received
0812 01:09:11.033598 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:11.033599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:11.033612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:11.033687 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:11.034012 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34086: BlockingRecv error: recv got EOF from 127.0.0.1:34086 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 01:09:11.034389 17928 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc)
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.035984 17928 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.037248 17928 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:09:11.046569 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:11.037930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34092 (local address 127.17.130.1:46245)
0812 01:09:11.038028 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:11.038031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:11.038122 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:09:11.038247 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:11.038248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:11.038267 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:11.038333 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:11.038756 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:11.039268 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:11.040004 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:11.040214 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:11.040291 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:11.042816 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:11.042817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:11.042818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:11.042825 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:11.044517 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:11.045563 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:11.045564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:11.045564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:11.045646 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:11.046000 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:11.046001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:11.046002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:11.046126 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:11.046220 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:11.046408 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 01:09:11.046495 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 01:09:11.049187 27431 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:11.048244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34098 (local address 127.17.130.1:46245)
0812 01:09:11.048314 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:11.048316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:11.048378 (+ 62us) server_negotiation.cc:407] Connection header received
0812 01:09:11.048457 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:11.048458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:11.048471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:11.048541 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:11.049119 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34098: BlockingRecv error: recv got EOF from 127.0.0.1:34098 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260812 01:09:11.049266 17928 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786496918848844-17928-0/krb5kdc/krb5cc' of type FILE
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.050580 17928 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:11.051637 17928 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[26620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.1@KRBTEST.COM
I20260812 01:09:11.061668 27431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:11.052305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34106 (local address 127.17.130.1:46245)
0812 01:09:11.052365 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:11.052366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:11.052468 (+ 102us) server_negotiation.cc:407] Connection header received
0812 01:09:11.052498 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:11.052499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:11.052510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:11.052571 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:11.053362 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:11.053839 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:11.055069 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:11.055272 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:11.055333 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:11.057718 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:11.057720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:11.057720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:11.057728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:11.059656 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:11.060154 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:11.060156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:11.060157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:11.060231 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:11.060590 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:11.060592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:11.060593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:11.060886 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:11.061266 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:11.061398 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 01:09:11.061500 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:09:11.062839 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 26794
I20260812 01:09:11.080955 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 27010
I20260812 01:09:11.100075 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 27210
I20260812 01:09:11.130019 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 26649
2026-08-12T01:09:11Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2420 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[27452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[27452](info): set up 2 sockets
Aug 12 01:09:11 dist-test-slave-tfpf krb5kdc[27452](info): commencing operation
krb5kdc: starting...
W20260812 01:09:12.496088 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:12 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab.
Entry for principal oryx/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab.
I20260812 01:09:12.641764 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:39477
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab
--principal=oryx/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:39477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:12.734529 27467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:12.734810 27467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:12.734853 27467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:12.736099 27467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:12.736176 27467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:12.736204 27467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:12.736236 27467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:12.736266 27467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:12.737986 27467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:39477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab
--principal=oryx/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:39477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:12.738317 27467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:12.738584 27467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:12.744164 27481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:12.744165 27477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:12.744256 27479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:12.744689 27467 server_base.cc:1034] running on GCE node
I20260812 01:09:12.744985 27467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:12.745371 27467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:12.746567 27467 hybrid_clock.cc:648] HybridClock initialized: now 1786496952746542 us; error 60 us; skew 500 ppm
Aug 12 01:09:12 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:12.749671 27467 init.cc:377] Logged in from keytab as oryx/127.17.130.62@KRBTEST.COM (short username oryx)
I20260812 01:09:12.750186 27467 webserver.cc:460] Webserver started at http://127.17.130.62:40941/ using document root <none> and password file <none>
I20260812 01:09:12.750454 27467 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:12.750527 27467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:12.750694 27467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:12.751842 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "3607476ecc8144b891de1f812c632ba7"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "5f847ad5646b085059c48e78b23d5565"
server_key_iv: "a3bcced5ce09f46c741df2b008091d60"
server_key_version: "encryptionkey@0"
I20260812 01:09:12.752249 27467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "3607476ecc8144b891de1f812c632ba7"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "5f847ad5646b085059c48e78b23d5565"
server_key_iv: "a3bcced5ce09f46c741df2b008091d60"
server_key_version: "encryptionkey@0"
I20260812 01:09:12.753865 27467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:12.754812 27492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:12.755168 27467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:12.755295 27467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "3607476ecc8144b891de1f812c632ba7"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "5f847ad5646b085059c48e78b23d5565"
server_key_iv: "a3bcced5ce09f46c741df2b008091d60"
server_key_version: "encryptionkey@0"
I20260812 01:09:12.755380 27467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:12.766443 27467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:12.769316 27467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:12.769505 27467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:12.774271 27467 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:39477
I20260812 01:09:12.774273 27585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:39477 every 8 connection(s)
I20260812 01:09:12.774788 27467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:12.776117 27586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:12.778713 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 27467
I20260812 01:09:12.778820 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:12.778762 27586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Bootstrap starting.
I20260812 01:09:12.779048 17928 external_mini_cluster.cc:1428] Setting key 75ae50ff4e41227a73eea45298177f4f
I20260812 01:09:12.779601 27586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:12.780074 27586 log.cc:824] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:12.780990 27586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: No bootstrap required, opened a new log
I20260812 01:09:12.783496 27586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } }
I20260812 01:09:12.783643 27586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:12.783667 27586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3607476ecc8144b891de1f812c632ba7, State: Initialized, Role: FOLLOWER
I20260812 01:09:12.783814 27586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } }
I20260812 01:09:12.783931 27586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:12.783957 27586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:12.784060 27586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:12.784673 27586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } }
I20260812 01:09:12.784802 27586 leader_election.cc:302] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3607476ecc8144b891de1f812c632ba7; no voters:
I20260812 01:09:12.785003 27586 leader_election.cc:288] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:12.785039 27592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:12.785182 27592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 LEADER]: Becoming Leader. State: Replica: 3607476ecc8144b891de1f812c632ba7, State: Running, Role: LEADER
I20260812 01:09:12.785274 27592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } }
I20260812 01:09:12.785358 27586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:12.785617 27592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3607476ecc8144b891de1f812c632ba7. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } }
I20260812 01:09:12.785633 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } }
I20260812 01:09:12.785801 27592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:12.785857 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:12.786140 27605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:12.786726 27605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 01:09:12 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:12.789106 27605 catalog_manager.cc:1347] Generated new cluster ID: 39ede93edfb342fbb55be0227a32b5e6
I20260812 01:09:12.789175 27605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:12.791852 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:12.779685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.17.130.62:39477)
0812 01:09:12.779936 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:12.779946 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:12.779988 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:09:12.780568 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:12.780575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:12.780674 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:12.780833 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:12.781616 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:12.783039 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:12.784062 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:12.784590 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:12.784728 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:12.788074 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:12.788078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:12.788078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:12.788091 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:12.790200 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:12.790618 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:12.790621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:12.790621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:12.790713 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:12.790953 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:12.790954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:12.790955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:12.791243 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:12.791411 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:12.791525 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 01:09:12.791677 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260812 01:09:12.803264 27605 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:12.804057 27605 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:12.807507 27605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Generated new TSK 0
I20260812 01:09:12.807736 27605 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab.
Entry for principal oryx/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab.
I20260812 01:09:12.848520 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39477
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:12.942548 27628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:12.942775 27628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:12.942802 27628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:12.944238 27628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:12.944301 27628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:12.944367 27628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:12.946125 27628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:12.946414 27628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:12.946682 27628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:12.952536 27637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:12.952561 27635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:12.952544 27634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:12.952981 27628 server_base.cc:1034] running on GCE node
I20260812 01:09:12.953267 27628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:12.953573 27628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:12.954739 27628 hybrid_clock.cc:648] HybridClock initialized: now 1786496952954698 us; error 55 us; skew 500 ppm
Aug 12 01:09:12 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:12.962494 27628 init.cc:377] Logged in from keytab as oryx/127.17.130.1@KRBTEST.COM (short username oryx)
I20260812 01:09:12.962984 27628 webserver.cc:460] Webserver started at http://127.17.130.1:32771/ using document root <none> and password file <none>
I20260812 01:09:12.963361 27628 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:12.963438 27628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:12.963544 27628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:12.964643 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "49d3da2d23ba4948ba554fa916cd061b"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "1d18a7a217c2ac3886c388501b781788"
server_key_iv: "9ed8142bcfe3667f380d0046af839dc3"
server_key_version: "encryptionkey@0"
I20260812 01:09:12.964967 27628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "49d3da2d23ba4948ba554fa916cd061b"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "1d18a7a217c2ac3886c388501b781788"
server_key_iv: "9ed8142bcfe3667f380d0046af839dc3"
server_key_version: "encryptionkey@0"
I20260812 01:09:12.967368 27628 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 01:09:12.968532 27654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:12.968743 27628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:12.968870 27628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "49d3da2d23ba4948ba554fa916cd061b"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "1d18a7a217c2ac3886c388501b781788"
server_key_iv: "9ed8142bcfe3667f380d0046af839dc3"
server_key_version: "encryptionkey@0"
I20260812 01:09:12.968967 27628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.004633 27628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:13.007766 27628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:13.007905 27628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:13.008249 27628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:13.008610 27628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:13.008669 27628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.008713 27628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:13.008741 27628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.016130 27628 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:36343
I20260812 01:09:13.016219 27856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:36343 every 8 connection(s)
I20260812 01:09:13.016592 27628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:13.018807 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 27628
I20260812 01:09:13.019033 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:13.019269 17928 external_mini_cluster.cc:1428] Setting key 37328d883de88612ace9a27a31523da2
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:13.028033 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.017572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:41809 (local address 127.17.130.62:39477)
0812 01:09:13.017643 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.017645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.017870 (+ 225us) server_negotiation.cc:407] Connection header received
0812 01:09:13.018400 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.018401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.018415 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.018492 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.019732 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.020447 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.021586 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.021931 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.022007 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.024679 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.024680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.024681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.024689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.026430 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.026917 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.026918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.026919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.026990 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.027271 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.027272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.027272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.027431 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.027514 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.027810 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.027932 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:09:13.028231 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.017499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:41809)
0812 01:09:13.017691 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.017752 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.018289 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.018614 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.018618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.019387 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.019650 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.019656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.020540 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.020541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.021477 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.021480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.022018 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.022019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.022153 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.022836 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.022847 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.024564 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.026560 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.026570 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.026573 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.026831 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.027066 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.027067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.027068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.027198 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.027504 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.027512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.027669 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.027786 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.027964 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260812 01:09:13.028971 27859 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39477
I20260812 01:09:13.029150 27859 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.17.130.2@KRBTEST.COM; defaulting to no policy
I20260812 01:09:13.029445 27859 heartbeater.cc:511] Master 127.17.130.62:39477 requested a full tablet report, sending...
I20260812 01:09:13.030287 27529 ts_manager.cc:194] Registered new tserver with Master: 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343)
I20260812 01:09:13.031136 27529 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:41809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab.
Entry for principal oryx/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab.
I20260812 01:09:13.052500 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39477
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:13.143880 27866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:13.144104 27866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:13.144129 27866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:13.145294 27866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:13.145340 27866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:13.145367 27866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:13.147168 27866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:13.147389 27866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:13.147594 27866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:13.153419 27873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.153424 27874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.153549 27876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:13.153762 27866 server_base.cc:1034] running on GCE node
I20260812 01:09:13.153999 27866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:13.154286 27866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:13.155498 27866 hybrid_clock.cc:648] HybridClock initialized: now 1786496953155493 us; error 42 us; skew 500 ppm
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:13.158860 27866 init.cc:377] Logged in from keytab as oryx/127.17.130.2@KRBTEST.COM (short username oryx)
I20260812 01:09:13.159539 27866 webserver.cc:460] Webserver started at http://127.17.130.2:33445/ using document root <none> and password file <none>
I20260812 01:09:13.159796 27866 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:13.159868 27866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:13.160024 27866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:13.161290 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "b7f9788d8806422dadbc726dadc79bd7"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "0ca2baab8c9f2aeebeae381d674a83ed"
server_key_iv: "f88babd0bbda75ba2cd25b5e54ada3dc"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.161688 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "b7f9788d8806422dadbc726dadc79bd7"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "0ca2baab8c9f2aeebeae381d674a83ed"
server_key_iv: "f88babd0bbda75ba2cd25b5e54ada3dc"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.163558 27866 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:13.164487 27885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.164757 27866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:13.164887 27866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "b7f9788d8806422dadbc726dadc79bd7"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "0ca2baab8c9f2aeebeae381d674a83ed"
server_key_iv: "f88babd0bbda75ba2cd25b5e54ada3dc"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.164973 27866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.173959 27866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:13.176683 27866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:13.176867 27866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:13.177203 27866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:13.177562 27866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:13.177620 27866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.177661 27866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:13.177693 27866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.184636 27866 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:39319
I20260812 01:09:13.184689 28067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:39319 every 8 connection(s)
I20260812 01:09:13.185094 27866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:13.190054 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 27866
I20260812 01:09:13.190286 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:13.190624 17928 external_mini_cluster.cc:1428] Setting key 26889081a6b500c4948412374d60a9c7
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:13.196206 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.185946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:33659)
0812 01:09:13.186140 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.186186 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.186762 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.187028 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.187032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.187668 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.187874 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.187879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.188604 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.188605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.189552 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.189555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.189964 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.189965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.190100 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.190803 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.190814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.192300 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.194516 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.194533 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.194536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.194801 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.195101 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.195102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.195102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.195243 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.195604 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.195612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.195760 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.195862 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.196010 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260812 01:09:13.196326 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.185999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:33659 (local address 127.17.130.62:39477)
0812 01:09:13.186072 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.186075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.186278 (+ 203us) server_negotiation.cc:407] Connection header received
0812 01:09:13.186883 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.186884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.186898 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.186952 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.187979 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.188498 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.189654 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.189873 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.189954 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.192404 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.192406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.192406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.192415 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.194404 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.194890 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.194891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.194891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.194975 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.195318 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.195319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.195319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.195484 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.195563 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.196063 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.196176 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:09:13.197099 28070 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39477
I20260812 01:09:13.197247 28070 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:13.197446 28070 heartbeater.cc:511] Master 127.17.130.62:39477 requested a full tablet report, sending...
I20260812 01:09:13.197896 27529 ts_manager.cc:194] Registered new tserver with Master: b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319)
I20260812 01:09:13.198349 27529 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:33659
WARNING: no policy specified for oryx/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab.
Entry for principal oryx/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab.
I20260812 01:09:13.225703 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39477
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:13.328279 28080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:13.328548 28080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:13.328580 28080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:13.330089 28080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:13.330166 28080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:13.330202 28080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:13.332202 28080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:13.332521 28080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:13.332806 28080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:13.339810 28090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.339807 28086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.339808 28088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:13.340884 28080 server_base.cc:1034] running on GCE node
I20260812 01:09:13.341192 28080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:13.341507 28080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:13.342765 28080 hybrid_clock.cc:648] HybridClock initialized: now 1786496953342731 us; error 31 us; skew 500 ppm
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:13.346529 28080 init.cc:377] Logged in from keytab as oryx/127.17.130.3@KRBTEST.COM (short username oryx)
I20260812 01:09:13.347304 28080 webserver.cc:460] Webserver started at http://127.17.130.3:40449/ using document root <none> and password file <none>
I20260812 01:09:13.347674 28080 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:13.347734 28080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:13.347962 28080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:13.349992 28080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "6bac5f7f6cb94b89916ba3a13462797e"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "f394c9763dcc61eac0419a67ab528d07"
server_key_iv: "bec8a68ef94282f171f5850195c1d63a"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.350453 28080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "6bac5f7f6cb94b89916ba3a13462797e"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "f394c9763dcc61eac0419a67ab528d07"
server_key_iv: "bec8a68ef94282f171f5850195c1d63a"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.352195 28080 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:09:13.353348 28107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.353742 28080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:13.353835 28080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "6bac5f7f6cb94b89916ba3a13462797e"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "f394c9763dcc61eac0419a67ab528d07"
server_key_iv: "bec8a68ef94282f171f5850195c1d63a"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.353904 28080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.367121 28080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:13.370116 28080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:13.370302 28080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:13.370631 28080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:13.371009 28080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:13.371071 28080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.371116 28080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:13.371146 28080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.378978 28080 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:37257
I20260812 01:09:13.379036 28286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:37257 every 8 connection(s)
I20260812 01:09:13.379525 28080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:13.383544 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 28080
I20260812 01:09:13.383668 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:13.383884 17928 external_mini_cluster.cc:1428] Setting key d9bee35c17e64bc0ea6bb04d8178a72d
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:13.391459 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.380545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:39437 (local address 127.17.130.62:39477)
0812 01:09:13.380623 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.380626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.381151 (+ 525us) server_negotiation.cc:407] Connection header received
0812 01:09:13.381821 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.381824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.381837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.381918 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.383171 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.384179 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.385542 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.385750 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.385840 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.388283 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.388285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.388285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.388297 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.389845 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.390356 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.390358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.390358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.390430 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.390726 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.390728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.390728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.390875 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.390963 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.391267 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.391359 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 01:09:13.391726 28290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.380531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:39437)
0812 01:09:13.380799 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.380842 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.381647 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.381994 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.381998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.382666 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.382892 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.382897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.384307 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.384309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.385431 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.385435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.385870 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.385871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.386008 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.386603 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.386612 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.388159 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.389980 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.389990 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.389993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.390265 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.390526 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.390527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.390527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.390652 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.390952 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.390960 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.391118 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.391232 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.391418 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":111,"threads_started":1}
I20260812 01:09:13.392398 28287 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39477
I20260812 01:09:13.392504 28287 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:13.392676 28287 heartbeater.cc:511] Master 127.17.130.62:39477 requested a full tablet report, sending...
I20260812 01:09:13.393261 27529 ts_manager.cc:194] Registered new tserver with Master: 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257)
I20260812 01:09:13.393718 27529 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:39437
I20260812 01:09:13.396644 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.61.keytab.
Entry for principal oryx/127.17.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.61.keytab.
I20260812 01:09:13.433483 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:40421
--webserver_interface=127.17.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.61.keytab
--principal=oryx/127.17.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:39477,127.17.130.61:40421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:13.532428 28295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:13.532706 28295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:13.532749 28295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:13.534152 28295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:13.534235 28295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:13.534263 28295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:13.534296 28295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:13.534330 28295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:13.536010 28295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:39477,127.17.130.61:40421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.61.keytab
--principal=oryx/127.17.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:40421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
W20260812 01:09:13.536206 28295 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.17.130.62:39477,127.17.130.61:40421'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 01:09:13.536371 28295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:13.536616 28295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:13.542613 28305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.542616 28302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.542747 28303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:13.543362 28295 server_base.cc:1034] running on GCE node
I20260812 01:09:13.543716 28295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:13.544070 28295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:13.545240 28295 hybrid_clock.cc:648] HybridClock initialized: now 1786496953545143 us; error 111 us; skew 500 ppm
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:13.548305 28295 init.cc:377] Logged in from keytab as oryx/127.17.130.61@KRBTEST.COM (short username oryx)
I20260812 01:09:13.548872 28295 webserver.cc:460] Webserver started at http://127.17.130.61:35759/ using document root <none> and password file <none>
I20260812 01:09:13.549211 28295 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:13.549284 28295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:13.549455 28295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:13.550678 28295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/instance:
uuid: "56987b3ad25742a2a267907febc3d2c0"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "d988c782e99d9dcc7962de8b568a2950"
server_key_iv: "99d518dfdda8c4cbc4998fb51f6f42f0"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.551083 28295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal/instance:
uuid: "56987b3ad25742a2a267907febc3d2c0"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "d988c782e99d9dcc7962de8b568a2950"
server_key_iv: "99d518dfdda8c4cbc4998fb51f6f42f0"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.552664 28295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:13.553776 28317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.554198 28295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:13.554319 28295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
uuid: "56987b3ad25742a2a267907febc3d2c0"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "d988c782e99d9dcc7962de8b568a2950"
server_key_iv: "99d518dfdda8c4cbc4998fb51f6f42f0"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.554412 28295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.578214 28295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:13.581668 28295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:13.581841 28295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:13.586581 28295 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.61:40421
I20260812 01:09:13.586629 28401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.61:40421 every 8 connection(s)
I20260812 01:09:13.587153 28295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
I20260812 01:09:13.588382 28403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:13.590207 28403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:09:13.591418 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 28295
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.61@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:13.601856 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.591151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.17.130.62:39477)
0812 01:09:13.591216 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.591219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.591542 (+ 323us) server_negotiation.cc:407] Connection header received
0812 01:09:13.592196 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.592197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.592211 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.592274 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.593075 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.593607 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.594777 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.594986 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.595080 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.597759 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.597761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.597761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.597770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.599665 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.600433 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.600435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.600435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.600508 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.600826 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.600827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.600828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.601106 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.601191 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.601576 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.601712 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:09:13.602041 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.590988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:35804)
0812 01:09:13.591369 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.591428 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.592069 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.592354 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.592358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.592777 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.592938 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.592943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.593728 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.593729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.594642 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.594647 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.595133 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.595135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.595263 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.595858 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.595867 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.597634 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.599875 (+ 2241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.599883 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.599886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.600238 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.600612 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.600613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.600614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.600738 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.601182 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.601189 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.601351 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.601529 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.601733 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":105,"thread_start_us":159,"threads_started":1}
I20260812 01:09:13.602838 28403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:09:13.603170 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.592097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55100 (local address 127.17.130.61:40421)
0812 01:09:13.592284 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.592288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.592451 (+ 163us) server_negotiation.cc:407] Connection header received
0812 01:09:13.592473 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.592475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.592509 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.592610 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.593270 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.594302 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.595383 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.595719 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.595818 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.598654 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.598664 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.598665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.598673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.600980 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.601484 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.601487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.601488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.601561 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.601848 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.601849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.601850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.602124 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.602824 (+ 700us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.602880 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.603063 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.61@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:13.611208 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.603242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.17.130.61:40421)
0812 01:09:13.603293 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.603295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.603500 (+ 205us) server_negotiation.cc:407] Connection header received
0812 01:09:13.603601 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.603602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.603611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.603670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.603978 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.604456 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.605312 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.605600 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.605675 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.608181 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.608182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.608183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.608191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.609957 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.610333 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.610334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.610335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.610394 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.610607 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.610608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.610608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.610732 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.610809 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.610954 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.611078 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":7,"spinlock_wait_cycles":768,"thread_start_us":105,"threads_started":1}
I20260812 01:09:13.611219 28407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.603204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:55108)
0812 01:09:13.603448 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.603460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.603523 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.603676 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.603677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.603838 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.603908 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.603910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.604576 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.604577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.605228 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.605231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.605677 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.605678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.605786 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.606324 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.606330 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.608054 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.610068 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.610069 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.610070 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.610245 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.610473 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.610474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.610474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.610538 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.610799 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.610800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.610842 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.610930 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.611128 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":97}
I20260812 01:09:13.613181 28403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Bootstrap starting.
I20260812 01:09:13.614133 28403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:13.614439 28403 log.cc:824] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:13.615387 28403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: No bootstrap required, opened a new log
I20260812 01:09:13.617427 28403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } }
I20260812 01:09:13.617573 28403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:13.617596 28403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56987b3ad25742a2a267907febc3d2c0, State: Initialized, Role: FOLLOWER
I20260812 01:09:13.617751 28403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } }
I20260812 01:09:13.618202 28403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:13.618175 28415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } } }
I20260812 01:09:13.618263 28415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:09:13.620523 28430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:09:13.620621 28430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:09:13.622658 28295 master_runner.cc:428] Detected that this master 56987b3ad25742a2a267907febc3d2c0 is joining an existing cluster
I20260812 01:09:13.622711 28295 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.130.61:40421
I20260812 01:09:13.631563 28439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.624113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:35818)
0812 01:09:13.624281 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.624290 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.624372 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.624536 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.624537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.624771 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.624860 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.624862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.625822 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.625823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.626397 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.626398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.626902 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.626903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.626973 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.627492 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.627499 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.627966 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.630215 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.630217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.630511 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.630754 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.630756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.630820 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.631121 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.631122 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.631169 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.631272 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.631409 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":2,"thread_start_us":97,"threads_started":1}
I20260812 01:09:13.631680 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.623922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.17.130.61:40421)
0812 01:09:13.624055 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.624057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.624205 (+ 148us) server_negotiation.cc:407] Connection header received
0812 01:09:13.624280 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.624281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.624290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.624361 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.624789 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.625570 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.626656 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.626934 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.626993 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.628265 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.628266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.628266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.628273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.630232 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630720 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.630721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.630721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.630778 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630991 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.630992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.630992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.631132 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.631216 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.631353 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.631505 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":80}
I20260812 01:09:13.632004 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.623908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:55110)
0812 01:09:13.624120 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.624129 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.624215 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.624364 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.624365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.624633 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.624704 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.624707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.625669 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.625670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.626558 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.626560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.626996 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.626997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.627070 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.627651 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.627657 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.628111 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.630325 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.630327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.630641 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.630838 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.630839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.630919 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.631205 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.631206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.631244 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.631378 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.631582 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":116,"threads_started":1}
I20260812 01:09:13.632427 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.623957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.17.130.62:39477)
0812 01:09:13.624043 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.624045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.624345 (+ 300us) server_negotiation.cc:407] Connection header received
0812 01:09:13.624431 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.624432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.624444 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.624501 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.624950 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.625738 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.626500 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.626805 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.626872 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.628082 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.628085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.628085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.628093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.630047 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630603 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.630604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.630604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.630677 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.630875 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.630876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.630876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.631031 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.631130 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.631316 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.631617 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.62@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:13.645953 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.635721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:55124)
0812 01:09:13.635950 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.636023 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.636155 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.636483 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.636487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.637038 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.637419 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.637424 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.638388 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.638389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.639041 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.639044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.639552 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.639553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.639623 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.640270 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.640275 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.641870 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.644237 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.644248 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.644251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.644492 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.644767 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.644768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.644768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.644843 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.645539 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.645548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.645608 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.645715 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.645848 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":78,"threads_started":1}
I20260812 01:09:13.645975 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.635942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.17.130.61:40421)
0812 01:09:13.636030 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.636033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.636141 (+ 108us) server_negotiation.cc:407] Connection header received
0812 01:09:13.636380 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.636381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.636393 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.636468 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.637529 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.638267 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.639138 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.639444 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.639529 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.641970 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.641972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.641972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.641980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.644086 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.644579 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.644581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.644581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.644653 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.645003 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.645007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.645007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.645419 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.645521 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.645768 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.645872 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 01:09:13.646751 27529 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.130.61:40421: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "56987b3ad25742a2a267907febc3d2c0"
member_type: NON_VOTER
last_known_addr {
host: "127.17.130.61"
port: 40421
}
attrs {
promote: true
}
}
dest_uuid: "3607476ecc8144b891de1f812c632ba7"
cas_config_opid_index: -1
I20260812 01:09:13.647245 27529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } }
I20260812 01:09:13.647897 28441 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:13.648252 28363 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:13.648954 28441 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.130.61:40421
I20260812 01:09:13.649055 28441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:13.649169 28441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:13.649363 28295 master.cc:537] Master@127.17.130.61:40421 shutting down...
I20260812 01:09:13.649729 28363 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 01:09:13.650136 28441 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 01:09:13.651206 27504 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 01:09:13.651693 27504 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 -> Peer 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61:40421): Couldn't send request to peer 56987b3ad25742a2a267907febc3d2c0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 01:09:13.652752 28295 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:13.652870 28295 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:13.652945 28295 tablet_replica.cc:330] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: stopping tablet replica
W20260812 01:09:13.654371 27449 connection.cc:535] client connection to 127.17.130.61:40421 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:09:13.666515 28295 master.cc:552] Master@127.17.130.61:40421 shutdown complete.
I20260812 01:09:13.666620 28295 master_runner.cc:305] Clearing existing system tablet
I20260812 01:09:13.666781 28295 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:13.666854 28295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:13.667589 28295 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.002s
I20260812 01:09:13.668244 28448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.668411 28295 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.668483 28295 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:13.668519 28295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
uuid: "56987b3ad25742a2a267907febc3d2c0"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "d988c782e99d9dcc7962de8b568a2950"
server_key_iv: "99d518dfdda8c4cbc4998fb51f6f42f0"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.669574 28295 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:13.671268 28295 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 01:09:13.671375 28295 log.cc:1196] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 01:09:13.671615 28295 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Deleting consensus metadata
I20260812 01:09:13.671751 28295 master_runner.cc:315] Copying system tablet from 127.17.130.62:39477
I20260812 01:09:13.672452 28295 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: tablet copy: Beginning tablet copy session from remote peer at address 127.17.130.62:39477
I20260812 01:09:13.681010 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.672926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.17.130.62:39477)
0812 01:09:13.672991 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.672992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.673108 (+ 116us) server_negotiation.cc:407] Connection header received
0812 01:09:13.673213 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.673214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.673228 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.673328 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.673709 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.674472 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.675587 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.675807 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.675920 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.677488 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.677490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.677491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.677503 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.679314 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.679798 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.679800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.679800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.679868 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.680080 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.680089 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.680089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.680231 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.680315 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.680688 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.680902 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 01:09:13.681042 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.672821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:35824)
0812 01:09:13.672985 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.672998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.673132 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.673341 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.673343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.673541 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.673617 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.673619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.674677 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.674680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.675470 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.675473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.675919 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.675921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.675997 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.676717 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.676725 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.677335 (+ 610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.679502 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.679504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.679504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.679713 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.679942 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.679943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.679943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.680012 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.680306 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.680307 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.680365 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.680624 (+ 259us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.680903 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":101,"threads_started":1}
I20260812 01:09:13.681563 27562 tablet_copy_service.cc:140] P 3607476ecc8144b891de1f812c632ba7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 56987b3ad25742a2a267907febc3d2c0 ({username='oryx', principal='oryx/127.17.130.61@KRBTEST.COM'} at 127.0.0.1:35824)
I20260812 01:09:13.681691 27562 tablet_copy_service.cc:161] P 3607476ecc8144b891de1f812c632ba7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 56987b3ad25742a2a267907febc3d2c0 at {username='oryx', principal='oryx/127.17.130.61@KRBTEST.COM'} at 127.0.0.1:35824: session id = 56987b3ad25742a2a267907febc3d2c0-00000000000000000000000000000000
I20260812 01:09:13.682518 27562 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 01:09:13.683526 28295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 01:09:13.685106 28295 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: tablet copy: Starting download of 0 data blocks...
I20260812 01:09:13.685279 28295 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: tablet copy: Starting download of 1 WAL segments...
I20260812 01:09:13.687145 28295 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 01:09:13.688328 27562 tablet_copy_service.cc:342] P 3607476ecc8144b891de1f812c632ba7: Request end of tablet copy session 56987b3ad25742a2a267907febc3d2c0-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.130.61@KRBTEST.COM'} at 127.0.0.1:35824
I20260812 01:09:13.688417 27562 tablet_copy_service.cc:433] P 3607476ecc8144b891de1f812c632ba7: ending tablet copy session 56987b3ad25742a2a267907febc3d2c0-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 56987b3ad25742a2a267907febc3d2c0
I20260812 01:09:13.691823 28295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:13.693332 28465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.693459 28463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.694209 28469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:13.694928 28295 server_base.cc:1034] running on GCE node
I20260812 01:09:13.695043 28295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:13.695191 28295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:13.696317 28295 hybrid_clock.cc:648] HybridClock initialized: now 1786496953696308 us; error 27 us; skew 500 ppm
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:13.698535 28295 init.cc:377] Logged in from keytab as oryx/127.17.130.61@KRBTEST.COM (short username oryx)
I20260812 01:09:13.699100 28295 webserver.cc:460] Webserver started at http://127.17.130.61:36567/ using document root <none> and password file <none>
I20260812 01:09:13.699275 28295 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:13.699323 28295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:13.706323 28295 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.001s
I20260812 01:09:13.708861 28481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.709048 28295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 01:09:13.709158 28295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
uuid: "56987b3ad25742a2a267907febc3d2c0"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "d988c782e99d9dcc7962de8b568a2950"
server_key_iv: "99d518dfdda8c4cbc4998fb51f6f42f0"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.709234 28295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.735198 28295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:13.737425 28295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:13.743160 28295 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.61:40421
I20260812 01:09:13.743273 28577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.61:40421 every 8 connection(s)
I20260812 01:09:13.743577 28295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
I20260812 01:09:13.744463 28578 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:09:13.744944 28578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Bootstrap starting.
I20260812 01:09:13.747363 28578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:09:13.747598 28578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: Bootstrap complete.
I20260812 01:09:13.747826 28578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } }
I20260812 01:09:13.747901 28578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 56987b3ad25742a2a267907febc3d2c0, State: Initialized, Role: LEARNER
I20260812 01:09:13.747929 28578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } }
I20260812 01:09:13.748126 28578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:13.748150 28584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:13.748257 28584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: LEARNER
I20260812 01:09:13.749879 28597 catalog_manager.cc:1259] Loaded cluster ID: 39ede93edfb342fbb55be0227a32b5e6
I20260812 01:09:13.750000 28597 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: loading cluster ID for follower catalog manager: success
I20260812 01:09:13.751004 28597 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: acquiring CA information for follower catalog manager: success
I20260812 01:09:13.751312 28597 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:09:13.766204 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.757023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.17.130.61:40421)
0812 01:09:13.757250 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.757252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.757299 (+ 47us) server_negotiation.cc:407] Connection header received
0812 01:09:13.757331 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.757332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.757345 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.757430 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.758227 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.759562 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.760683 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.761004 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.761146 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.762585 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.762587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.762587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.762595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.764529 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.764921 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.764922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.764923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.764995 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.765369 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.765370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.765370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.765534 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.765637 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.765852 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.765949 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for oryx/127.17.130.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.60.keytab.
Entry for principal oryx/127.17.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.60.keytab.
I20260812 01:09:13.802367 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:45807
--webserver_interface=127.17.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.60.keytab
--principal=oryx/127.17.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:13.903407 28605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:13.903679 28605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:13.903707 28605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:13.905331 28605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:13.905467 28605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:13.905489 28605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:13.905506 28605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:13.905522 28605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:13.908851 28605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.60.keytab
--principal=oryx/127.17.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:13.909354 28605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:13.909669 28605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:13.915297 28616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.915549 28619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:13.915568 28614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:13.915711 28605 server_base.cc:1034] running on GCE node
I20260812 01:09:13.915992 28605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:13.916288 28605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:13.917546 28605 hybrid_clock.cc:648] HybridClock initialized: now 1786496953917536 us; error 45 us; skew 500 ppm
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:13.921713 28605 init.cc:377] Logged in from keytab as oryx/127.17.130.60@KRBTEST.COM (short username oryx)
I20260812 01:09:13.922487 28605 webserver.cc:460] Webserver started at http://127.17.130.60:44829/ using document root <none> and password file <none>
I20260812 01:09:13.922950 28605 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:13.923017 28605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:13.923198 28605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:13.924736 28605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/instance:
uuid: "17378047f95b45378026e0dc4e458466"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "15256c3c382258b95047dbc5a9fa1b3a"
server_key_iv: "32927a48551355fce0d3f8fe0aec0603"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.925328 28605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal/instance:
uuid: "17378047f95b45378026e0dc4e458466"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "15256c3c382258b95047dbc5a9fa1b3a"
server_key_iv: "32927a48551355fce0d3f8fe0aec0603"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.927198 28605 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 01:09:13.928411 28632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:13.928587 28605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:13.928691 28605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
uuid: "17378047f95b45378026e0dc4e458466"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "15256c3c382258b95047dbc5a9fa1b3a"
server_key_iv: "32927a48551355fce0d3f8fe0aec0603"
server_key_version: "encryptionkey@0"
I20260812 01:09:13.928793 28605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:13.948122 28605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:13.951655 28605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:13.951879 28605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:13.957165 28605 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.60:45807
I20260812 01:09:13.957320 28723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.60:45807 every 8 connection(s)
I20260812 01:09:13.957682 28605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
I20260812 01:09:13.959281 28725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:13.960521 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 28605
I20260812 01:09:13.963019 28725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.60@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.130.60@KRBTEST.COM
I20260812 01:09:13.974866 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.964389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.17.130.62:39477)
0812 01:09:13.964465 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.964467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.964642 (+ 175us) server_negotiation.cc:407] Connection header received
0812 01:09:13.964748 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.964749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.964762 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.964840 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.965927 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.966455 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.967432 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.967649 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.967764 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.970628 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.970630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.970631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.970642 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.972735 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.973453 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.973454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.973455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.973588 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.973920 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.973922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.973922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.974112 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.974197 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.974584 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.974730 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:09:13.975041 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.964245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:35840)
0812 01:09:13.964505 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.964564 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.964669 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.964882 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.964885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.965329 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.965683 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.965696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.966576 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.966577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.967327 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.967330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.967801 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.967803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.967878 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.968474 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.968484 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.970427 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.972881 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.972891 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.972894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.973358 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.973680 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.973681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.973688 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.973825 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.974204 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.974210 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.974352 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.974488 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.974759 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":148,"threads_started":1}
I20260812 01:09:13.975541 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.961570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.17.130.60:45807)
0812 01:09:13.961860 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.961979 (+ 119us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.962032 (+ 53us) server_negotiation.cc:407] Connection header received
0812 01:09:13.963052 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.963066 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.963163 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.963585 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.964619 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.965913 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.966748 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.967251 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.967398 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.971148 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.971158 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.971159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.971167 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.974004 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.974430 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.974434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.974435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.974503 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.974809 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.974810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.974810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.974976 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.975121 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.975348 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.975431 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260812 01:09:13.975857 28725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.60@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:13.983848 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.976424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:55144)
0812 01:09:13.976483 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.976495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.976539 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.976788 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.976789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.976938 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.976990 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.976992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.977704 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.977705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.978232 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.978234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.978633 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.978634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.978734 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.979218 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.979223 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.980596 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.982705 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.982706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.982707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.982890 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.983127 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.983127 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.983128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.983193 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.983466 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.983467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.983544 (+ 77us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.983624 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.983744 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:09:13.983848 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.976552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55144 (local address 127.17.130.61:40421)
0812 01:09:13.976651 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.976653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.976661 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:13.976697 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.976698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.976711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.976783 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.977101 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.977612 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.978334 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.978548 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.978629 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.980715 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.980717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.980717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.980725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.982554 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.982988 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.982989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.982989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.983055 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.983262 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.983263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.983264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.983393 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.983475 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.983654 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.983746 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 01:09:13.984772 28725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:09:13 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496953, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.60@KRBTEST.COM for oryx/127.17.130.60@KRBTEST.COM
I20260812 01:09:13.992159 28734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.985215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.0.0.1:45314)
0812 01:09:13.985272 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:13.985285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:13.985341 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:13.985505 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:13.985507 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:13.985658 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:13.985697 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.985698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.986447 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.986448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.987001 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:13.987002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.987465 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.987468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:13.987561 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.988246 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:13.988252 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:13.989548 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:13.991046 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.991047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.991048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.991219 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.991428 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:13.991429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:13.991429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:13.991564 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:13.991835 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:13.991835 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:13.991875 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 01:09:13.991932 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:09:13.992028 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:09:13.992172 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:13.985226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.17.130.60:45807)
0812 01:09:13.985275 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:13.985277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:13.985323 (+ 46us) server_negotiation.cc:407] Connection header received
0812 01:09:13.985399 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:13.985400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:13.985410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:13.985481 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:13.985779 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.986342 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.987112 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:13.987320 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:13.987390 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:13.989636 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:13.989637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:13.989638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:13.989645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:13.990933 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.991291 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.991292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.991292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.991346 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:13.991644 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:13.991644 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:13.991644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:13.991774 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:13.991827 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:13.991966 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 01:09:13.992078 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":26}
I20260812 01:09:13.993809 28725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Bootstrap starting.
I20260812 01:09:13.994858 28725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:13.995208 28725 log.cc:824] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:13.996428 28725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: No bootstrap required, opened a new log
I20260812 01:09:13.998353 28725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } }
I20260812 01:09:13.998503 28725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:13.998526 28725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17378047f95b45378026e0dc4e458466, State: Initialized, Role: FOLLOWER
I20260812 01:09:13.998673 28725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } }
I20260812 01:09:13.999060 28725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:13.999074 28739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } } }
I20260812 01:09:13.999213 28739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:09:14.001292 28762 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:09:14.001371 28762 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:09:14.004483 28605 master_runner.cc:428] Detected that this master 17378047f95b45378026e0dc4e458466 is joining an existing cluster
I20260812 01:09:14.004550 28605 master_runner.cc:432] Initiating AddMaster RPC to add 127.17.130.60:45807
I20260812 01:09:14.014288 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.005860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.17.130.62:39477)
0812 01:09:14.005912 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.005914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.006290 (+ 376us) server_negotiation.cc:407] Connection header received
0812 01:09:14.006347 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.006349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.006363 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.006430 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.006999 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.007808 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.008916 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.009256 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.009335 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.010803 (+ 1468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.010804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.010804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.010814 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.012606 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.013344 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.013346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.013346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.013428 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.013650 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.013651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.013652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.013791 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.013872 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.014087 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.014180 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:09:14.014308 28770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.005780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:35856)
0812 01:09:14.006203 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.006215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.006283 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.006530 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.006531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.006809 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.006894 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.006897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.007916 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.007917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.008815 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.008818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.009392 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.009392 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.009467 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.010008 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.010016 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.010691 (+ 675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.012787 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.012789 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.012790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.013228 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.013507 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.013508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.013508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.013581 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.013872 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.013873 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.013933 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.014051 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.014181 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":97,"threads_started":1}
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496952, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.62@KRBTEST.COM for oryx/127.17.130.60@KRBTEST.COM
I20260812 01:09:14.027040 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 27628
I20260812 01:09:14.045986 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.005968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.17.130.61:40421)
0812 01:09:14.006045 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.006047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.006433 (+ 386us) server_negotiation.cc:407] Connection header received
0812 01:09:14.006546 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.006547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.006559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.006591 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.007030 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.007547 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.008335 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.008576 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.008640 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.041620 (+ 32980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.041623 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.041623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.041635 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.043444 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.044841 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.044842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.044843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.044912 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.045367 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.045368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.045368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.045522 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.045569 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.045794 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.045892 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:09:14.046391 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.005781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:55158)
0812 01:09:14.006371 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.006381 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.006457 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.006666 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.006667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.006880 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.006931 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.006933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.007644 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.007645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.008244 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.008246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.008714 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.008715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.008766 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.009359 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.009365 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.041457 (+ 32092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.043560 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.043561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.043562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.044286 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.044993 (+ 707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.044994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.044994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.045084 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.045634 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.045635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.045683 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.046147 (+ 464us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.046271 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":3,"thread_start_us":104,"threads_started":1}
I20260812 01:09:14.047829 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.017770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.0.0.1:45334)
0812 01:09:14.017835 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.017845 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.017879 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.018180 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.018182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.018362 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.018408 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.018410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.019157 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.019158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.019685 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.019688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.020104 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.020106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.020182 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.020736 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.020741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.022779 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.046542 (+ 23763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.046544 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.046544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.046804 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.047060 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.047061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.047061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.047129 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.047417 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.047418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.047488 (+ 70us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.047549 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.047675 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 01:09:14.049594 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.006059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.17.130.60:45807)
0812 01:09:14.006100 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.006101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.006430 (+ 329us) server_negotiation.cc:407] Connection header received
0812 01:09:14.006519 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.006519 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.006529 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.006557 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.040641 (+ 34084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.041393 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.043026 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.043336 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.043381 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.046245 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.046246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.046246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.046254 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.047917 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.048819 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.048820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.048821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.048868 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.049039 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.049039 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.049040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.049195 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.049241 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.049412 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.049488 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:09:14.049791 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:36343
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:39477
--webserver_port=32771
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:09:14.049860 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.006107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.0.0.1:45318)
0812 01:09:14.006381 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.006389 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.006450 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.006614 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.006615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.040363 (+ 33748us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.040474 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.040480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.041503 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.041504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.042910 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.042916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.043461 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.043461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.043540 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.044716 (+ 1176us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.044722 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.045364 (+ 642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.048514 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.048514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.048515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.048739 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.048929 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.048930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.048930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.048977 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.049282 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.049283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.049317 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.049671 (+ 354us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.049764 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":169,"threads_started":1}
I20260812 01:09:14.050230 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.017827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.17.130.60:45807)
0812 01:09:14.018038 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.018039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.018046 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:09:14.018079 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.018080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.018089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.018163 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.018497 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.019054 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.019776 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.020021 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.020106 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.022889 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.022892 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.022892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.022903 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.046126 (+ 23223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.046896 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.046897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.046897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.046976 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.047201 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.047202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.047203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.047341 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.050015 (+ 2674us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.050055 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.050132 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":122,"threads_started":1}
I20260812 01:09:14.050680 27529 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.17.130.60:45807: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "17378047f95b45378026e0dc4e458466"
member_type: NON_VOTER
last_known_addr {
host: "127.17.130.60"
port: 45807
}
attrs {
promote: true
}
}
dest_uuid: "3607476ecc8144b891de1f812c632ba7"
cas_config_opid_index: 5
I20260812 01:09:14.050855 27529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } }
I20260812 01:09:14.051378 28441 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 17378047f95b45378026e0dc4e458466 (127.17.130.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.052248 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.044679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:55162)
0812 01:09:14.047917 (+ 3238us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.047929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.047984 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.048295 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.048296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.048298 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.048348 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.048350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.049320 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.049321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.050526 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.050530 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.051990 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.051991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.052058 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.052083 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.052135 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.052178 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3225,"thread_start_us":119,"threads_started":1}
I20260812 01:09:14.052382 28441 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.17.130.60:45807
I20260812 01:09:14.052443 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.044911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.17.130.61:40421)
0812 01:09:14.046089 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.046091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.048060 (+ 1969us) server_negotiation.cc:407] Connection header received
0812 01:09:14.048088 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.048089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.048136 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.048205 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.048444 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.049207 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.051080 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.051763 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.051849 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.051945 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.052332 (+ 387us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.052379 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1162,"thread_start_us":77,"threads_started":1}
I20260812 01:09:14.052464 28441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.052531 28441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:14.052991 28533 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 01:09:14.053174 28678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:14.053614 28531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 LEARNER]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 01:09:14.053699 28584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3607476ecc8144b891de1f812c632ba7. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:14.053761 28584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: LEARNER
I20260812 01:09:14.054118 28782 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 01:09:14.054338 28678 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 01:09:14.054683 28442 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:14.054698 28531 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 17378047f95b45378026e0dc4e458466 (127.17.130.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.055784 28584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.055872 28584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: LEARNER
I20260812 01:09:14.056393 28678 raft_consensus.cc:858] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } }
W20260812 01:09:14.056478 28678 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } }
I20260812 01:09:14.056521 28442 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: attempting to promote NON_VOTER 56987b3ad25742a2a267907febc3d2c0 to VOTER
I20260812 01:09:14.056744 28442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } }
I20260812 01:09:14.057487 28678 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:14.057654 28531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 LEARNER]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 01:09:14.058009 28441 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 01:09:14.058693 28441 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.059549 28739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3607476ecc8144b891de1f812c632ba7. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:14.059659 28739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 01:09:14.059674 28441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.059710 28739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:14.059764 28441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:14.059767 28739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 01:09:14.059805 28739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: NON_VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: true } } }
I20260812 01:09:14.059855 28739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 01:09:14.059984 28782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.060053 28782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:14.060284 28605 master.cc:537] Master@127.17.130.60:45807 shutting down...
I20260812 01:09:14.060277 28531 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.061877 28786 mvcc.cc:205] Tried to move back new op lower bound from 7317491518697414656 to 7317491518608801792. Current Snapshot: MvccSnapshot[applied={T|T < 7317491518619381760}]
I20260812 01:09:14.061971 28584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.062083 28584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:09:14.062543 28677 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260812 01:09:14.062922 28677 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 01:09:14.063591 28782 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260812 01:09:14.064347 27504 consensus_peers.cc:487] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 -> Peer 17378047f95b45378026e0dc4e458466 (127.17.130.60:45807): Couldn't send request to peer 17378047f95b45378026e0dc4e458466. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 01:09:14.064981 28605 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 01:09:14.065146 28605 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 01:09:14.065200 28605 tablet_replica.cc:330] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: stopping tablet replica
I20260812 01:09:14.069378 28605 master.cc:552] Master@127.17.130.60:45807 shutdown complete.
I20260812 01:09:14.069440 28605 master_runner.cc:305] Clearing existing system tablet
I20260812 01:09:14.069587 28605 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:14.069643 28605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:14.070465 28605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:14.071192 28802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.071378 28605 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:14.071506 28605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:14.071550 28605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
uuid: "17378047f95b45378026e0dc4e458466"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "15256c3c382258b95047dbc5a9fa1b3a"
server_key_iv: "32927a48551355fce0d3f8fe0aec0603"
server_key_version: "encryptionkey@0"
I20260812 01:09:14.072454 28605 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:14.074119 28605 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 01:09:14.074234 28605 log.cc:1196] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 01:09:14.074545 28605 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Deleting consensus metadata
I20260812 01:09:14.074717 28605 master_runner.cc:315] Copying system tablet from 127.17.130.62:39477
I20260812 01:09:14.075445 28605 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: tablet copy: Beginning tablet copy session from remote peer at address 127.17.130.62:39477
I20260812 01:09:14.083588 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.075908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35868 (local address 127.17.130.62:39477)
0812 01:09:14.075964 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.075966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.076244 (+ 278us) server_negotiation.cc:407] Connection header received
0812 01:09:14.076304 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.076305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.076316 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.076391 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.076837 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.077671 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.078704 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.079035 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.079123 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.080328 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.080329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.080330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.080338 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.082259 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.082698 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.082699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.082700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.082768 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.082993 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.082993 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.082994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.083151 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.083223 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.083401 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.083487 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:09:14.083616 28812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.075829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:35868)
0812 01:09:14.076152 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.076166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.076244 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.076404 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.076405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.076665 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.076750 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.076753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.077771 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.077771 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.078600 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.078603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.079138 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.079139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.079217 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.079791 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.079797 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.080224 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.082358 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.082360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.082360 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.082611 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.082845 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.082845 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.082846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.082916 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.083231 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.083232 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.083285 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.083360 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.083483 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":77,"threads_started":1}
I20260812 01:09:14.084224 27562 tablet_copy_service.cc:140] P 3607476ecc8144b891de1f812c632ba7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 17378047f95b45378026e0dc4e458466 ({username='oryx', principal='oryx/127.17.130.60@KRBTEST.COM'} at 127.0.0.1:35868)
I20260812 01:09:14.084286 27562 tablet_copy_service.cc:161] P 3607476ecc8144b891de1f812c632ba7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 17378047f95b45378026e0dc4e458466 at {username='oryx', principal='oryx/127.17.130.60@KRBTEST.COM'} at 127.0.0.1:35868: session id = 17378047f95b45378026e0dc4e458466-00000000000000000000000000000000
I20260812 01:09:14.084884 27562 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 01:09:14.085646 28605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 01:09:14.087234 28605 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: tablet copy: Starting download of 0 data blocks...
I20260812 01:09:14.087494 28605 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: tablet copy: Starting download of 1 WAL segments...
I20260812 01:09:14.088752 28605 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 01:09:14.089888 27562 tablet_copy_service.cc:342] P 3607476ecc8144b891de1f812c632ba7: Request end of tablet copy session 17378047f95b45378026e0dc4e458466-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.17.130.60@KRBTEST.COM'} at 127.0.0.1:35868
I20260812 01:09:14.089948 27562 tablet_copy_service.cc:433] P 3607476ecc8144b891de1f812c632ba7: ending tablet copy session 17378047f95b45378026e0dc4e458466-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 17378047f95b45378026e0dc4e458466
I20260812 01:09:14.093763 28605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:14.095264 28819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.098189 28821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.098222 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:14.098418 28605 server_base.cc:1034] running on GCE node
I20260812 01:09:14.098737 28605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:14.098909 28605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:14.100051 28605 hybrid_clock.cc:648] HybridClock initialized: now 1786496954100023 us; error 45 us; skew 500 ppm
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:14.101917 28605 init.cc:377] Logged in from keytab as oryx/127.17.130.60@KRBTEST.COM (short username oryx)
I20260812 01:09:14.102243 28605 webserver.cc:460] Webserver started at http://127.17.130.60:34885/ using document root <none> and password file <none>
I20260812 01:09:14.102370 28605 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:14.102407 28605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:14.103473 28605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:14.104264 28838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.104435 28605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:14.104503 28605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
uuid: "17378047f95b45378026e0dc4e458466"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "15256c3c382258b95047dbc5a9fa1b3a"
server_key_iv: "32927a48551355fce0d3f8fe0aec0603"
server_key_version: "encryptionkey@0"
I20260812 01:09:14.104591 28605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:14.129308 28605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:14.131642 28605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:14.135500 28605 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.60:45807
I20260812 01:09:14.135948 28605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
I20260812 01:09:14.137054 28930 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:09:14.137663 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Bootstrap starting.
I20260812 01:09:14.138952 28928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.60:45807 every 8 connection(s)
I20260812 01:09:14.140170 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:09:14.140434 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Bootstrap complete.
I20260812 01:09:14.140654 28930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } }
I20260812 01:09:14.140750 28930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 17378047f95b45378026e0dc4e458466, State: Initialized, Role: LEARNER
I20260812 01:09:14.140786 28930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } }
I20260812 01:09:14.140964 28930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:14.141176 28933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.141374 28933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: LEARNER
I20260812 01:09:14.142676 28950 catalog_manager.cc:1259] Loaded cluster ID: 39ede93edfb342fbb55be0227a32b5e6
I20260812 01:09:14.142786 28950 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: loading cluster ID for follower catalog manager: success
I20260812 01:09:14.144665 28950 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: acquiring CA information for follower catalog manager: success
I20260812 01:09:14.145008 28950 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 01:09:14.168424 28783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:14.168771 28783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:14.168818 28783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:14.170382 28783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:14.170487 28783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:14.170543 28783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:14.172297 28783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:36343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=32771
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:14.172643 28783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:14.172923 28783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:14.179942 28961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.179945 28958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.179945 28956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:14.180442 28783 server_base.cc:1034] running on GCE node
I20260812 01:09:14.180680 28783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:14.180980 28783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:14.182283 28783 hybrid_clock.cc:648] HybridClock initialized: now 1786496954182253 us; error 51 us; skew 500 ppm
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:14.186383 28783 init.cc:377] Logged in from keytab as oryx/127.17.130.1@KRBTEST.COM (short username oryx)
I20260812 01:09:14.186981 28783 webserver.cc:460] Webserver started at http://127.17.130.1:32771/ using document root <none> and password file <none>
I20260812 01:09:14.187245 28783 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:14.187328 28783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:14.189451 28783 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:14.190394 28978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.190711 28783 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:14.190821 28783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "49d3da2d23ba4948ba554fa916cd061b"
format_stamp: "Formatted at 2026-08-12 01:09:12 on dist-test-slave-tfpf"
server_key: "1d18a7a217c2ac3886c388501b781788"
server_key_iv: "9ed8142bcfe3667f380d0046af839dc3"
server_key_version: "encryptionkey@0"
I20260812 01:09:14.191329 28783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:14.199412 28070 heartbeater.cc:503] Master 127.17.130.62:39477 was elected leader, sending a full tablet report...
I20260812 01:09:14.217952 28783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:14.220733 28783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:14.220867 28783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:14.221362 28783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:14.221717 28783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:14.221755 28783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.221778 28783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:14.221801 28783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.230909 28783 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:36343
I20260812 01:09:14.231071 29154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:36343 every 8 connection(s)
I20260812 01:09:14.231451 28783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:14.239948 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 28783
I20260812 01:09:14.240139 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 27866
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:14.260582 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.232869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:52467 (local address 127.17.130.61:40421)
0812 01:09:14.232920 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.232923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.233032 (+ 109us) server_negotiation.cc:407] Connection header received
0812 01:09:14.234154 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.234155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.234173 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.234216 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.235349 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.235875 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237522 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.237766 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237873 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.256033 (+ 18160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.256036 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.256036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.256046 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.258701 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.259366 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.259368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.259368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.259428 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.259680 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.259680 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.259681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.259864 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.259917 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.260342 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.260446 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:09:14.261905 29169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.232706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.17.130.1:52467)
0812 01:09:14.232943 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.232953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.233990 (+ 1037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.234475 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.234475 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.234902 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.235214 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.235221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.236003 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.236004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.237417 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.237419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.237889 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.238124 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.239513 (+ 1389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.239518 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.255872 (+ 16354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.258859 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.258875 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.258877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.259287 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.259487 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.259487 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.259488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.259619 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.259969 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.259976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.260196 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.261222 (+ 1026us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.261610 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":131,"thread_start_us":77,"threads_started":1}
I20260812 01:09:14.263692 29159 heartbeater.cc:347] Connected to a master server at 127.17.130.61:40421
I20260812 01:09:14.263835 29159 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.264068 29159 heartbeater.cc:511] Master 127.17.130.61:40421 requested a full tablet report, sending...
I20260812 01:09:14.264791 28510 ts_manager.cc:194] Registered new tserver with Master: 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343)
I20260812 01:09:14.269272 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.232679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47891 (local address 127.17.130.62:39477)
0812 01:09:14.232741 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.232743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.232809 (+ 66us) server_negotiation.cc:407] Connection header received
0812 01:09:14.234107 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.234109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.234126 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.234308 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.235994 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.236483 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237537 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.237837 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237901 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.265260 (+ 27359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.265263 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.265264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.265276 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.267425 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.267987 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.267989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.267989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.268062 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.268371 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.268372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.268373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.268570 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.268644 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.268950 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.269101 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:09:14.269762 29162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.232382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:47891)
0812 01:09:14.232625 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.232685 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.233942 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.234403 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.234435 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.234872 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.235215 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.235222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.236595 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.236597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.237430 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.237433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.237975 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.238107 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.239292 (+ 1185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.239302 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.265054 (+ 25752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.267558 (+ 2504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.267560 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.267560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.267866 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.268152 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.268153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.268153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.268256 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.268688 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.268690 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.268756 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.269452 (+ 696us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.269603 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":52,"thread_start_us":110,"threads_started":1}
I20260812 01:09:14.270117 29158 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39477
I20260812 01:09:14.270193 29158 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.270298 29158 heartbeater.cc:511] Master 127.17.130.62:39477 requested a full tablet report, sending...
I20260812 01:09:14.270581 27529 ts_manager.cc:194] Re-registered known tserver with Master: 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343)
I20260812 01:09:14.271142 27529 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:47891
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.60@KRBTEST.COM
I20260812 01:09:14.275046 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.232433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:37783 (local address 127.17.130.60:45807)
0812 01:09:14.232601 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.232604 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.232897 (+ 293us) server_negotiation.cc:407] Connection header received
0812 01:09:14.234098 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.234100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.234127 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.234220 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.235342 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.236091 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237519 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.237838 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237882 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.271888 (+ 34006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.271891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.271891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.271902 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.273698 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.274170 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.274171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.274172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.274241 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.274491 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.274492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.274492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.274649 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.274711 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.274893 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.274973 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260812 01:09:14.275187 29165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.232382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.17.130.1:37783)
0812 01:09:14.232681 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.232691 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.233942 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.234402 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.234407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.234914 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.235214 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.235221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.236197 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.236198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.237403 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.237407 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.237985 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.237986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.238107 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.239388 (+ 1281us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.239398 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.271749 (+ 32351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.273799 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.273801 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.273802 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.274056 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.274312 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.274313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.274313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.274388 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.274723 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.274724 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.274777 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.274846 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.275000 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":55,"thread_start_us":106,"threads_started":1}
I20260812 01:09:14.275883 29156 heartbeater.cc:347] Connected to a master server at 127.17.130.60:45807
I20260812 01:09:14.275972 29156 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.276101 29156 heartbeater.cc:511] Master 127.17.130.60:45807 requested a full tablet report, sending...
I20260812 01:09:14.276573 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:39319
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:39477
--webserver_port=33445
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:09:14.276611 28877 ts_manager.cc:194] Registered new tserver with Master: 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343)
W20260812 01:09:14.373172 29170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:14.373407 29170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:14.373433 29170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:14.374607 29170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:14.374650 29170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:14.374678 29170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:14.376605 29170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:39319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=33445
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:14.376848 29170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:14.377043 29170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:14.383828 29180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.383839 29177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.383832 29184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:14.384193 29170 server_base.cc:1034] running on GCE node
I20260812 01:09:14.384369 29170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:14.384622 29170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:14.385850 29170 hybrid_clock.cc:648] HybridClock initialized: now 1786496954385835 us; error 33 us; skew 500 ppm
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:14.389106 29170 init.cc:377] Logged in from keytab as oryx/127.17.130.2@KRBTEST.COM (short username oryx)
I20260812 01:09:14.389669 29170 webserver.cc:460] Webserver started at http://127.17.130.2:33445/ using document root <none> and password file <none>
I20260812 01:09:14.389919 29170 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:14.389987 29170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:14.391665 29170 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:14.392479 29196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.392715 29170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:14.392840 29170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "b7f9788d8806422dadbc726dadc79bd7"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "0ca2baab8c9f2aeebeae381d674a83ed"
server_key_iv: "f88babd0bbda75ba2cd25b5e54ada3dc"
server_key_version: "encryptionkey@0"
I20260812 01:09:14.393291 29170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:14.395414 28287 heartbeater.cc:503] Master 127.17.130.62:39477 was elected leader, sending a full tablet report...
I20260812 01:09:14.401475 29170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:14.404073 29170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:14.404273 29170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:14.404495 29170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:14.404850 29170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:14.404907 29170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.404944 29170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:14.404994 29170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.412744 29170 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:39319
I20260812 01:09:14.412919 29383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:39319 every 8 connection(s)
I20260812 01:09:14.413427 29170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.60@KRBTEST.COM
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:14.423696 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 29170
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:14.424131 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 28080
I20260812 01:09:14.435209 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.415063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:33403 (local address 127.17.130.61:40421)
0812 01:09:14.415114 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.415116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.415144 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:09:14.416087 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.416088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.416103 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.416170 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.418513 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.420155 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.421760 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.422201 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.422290 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.428947 (+ 6657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.428949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.428949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.428959 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.432745 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.433352 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.433355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.433356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.433420 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.433707 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.433707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.433708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.433845 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.434045 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.434639 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.435028 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:09:14.435352 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.414920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.17.130.2:33403)
0812 01:09:14.415087 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.415096 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.415967 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.416261 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.416265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.416776 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.417078 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.417085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.420756 (+ 3671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.420756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.421656 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.421659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.422311 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.422313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.422385 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.422810 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.422815 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.428809 (+ 5994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.432869 (+ 4060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.432891 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.432895 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.433254 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.433493 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.433494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.433494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.433626 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.434099 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.434119 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.434379 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.434581 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.435037 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":25,"thread_start_us":71,"threads_started":1}
I20260812 01:09:14.445933 29388 heartbeater.cc:347] Connected to a master server at 127.17.130.61:40421
I20260812 01:09:14.446074 29388 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.446272 29388 heartbeater.cc:511] Master 127.17.130.61:40421 requested a full tablet report, sending...
I20260812 01:09:14.446844 28510 ts_manager.cc:194] Registered new tserver with Master: b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319)
I20260812 01:09:14.451969 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.414701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:41473 (local address 127.17.130.62:39477)
0812 01:09:14.414778 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.414780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.415093 (+ 313us) server_negotiation.cc:407] Connection header received
0812 01:09:14.416177 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.416179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.416192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.416249 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.417413 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.417896 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.420310 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.420603 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.420695 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.423241 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.423243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.423244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.423252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.450566 (+ 27314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.451049 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.451051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.451051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.451129 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.451360 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.451361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.451361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.451503 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.451580 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.451771 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.451870 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:09:14.452167 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:37257
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:39477
--webserver_port=40449
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:44249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:09:14.452504 29393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.414614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:41473)
0812 01:09:14.414980 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.415027 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.415987 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.416277 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.416278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.416791 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.417151 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.417155 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.418107 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.418108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.420138 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.420149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.420712 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.420713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.420808 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.421378 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.421383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.423098 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.450699 (+ 27601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.450701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.450702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.450937 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.451207 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.451208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.451208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.451281 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.451584 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.451585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.451643 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.451742 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.451913 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260812 01:09:14.453161 29386 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39477
I20260812 01:09:14.453187 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.414883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:51971 (local address 127.17.130.60:45807)
0812 01:09:14.414946 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.414948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.414960 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:09:14.416270 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.416272 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.416284 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.416344 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.417209 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.418269 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.419441 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.419673 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.419764 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.423201 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.423202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.423202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.423211 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.451507 (+ 28296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.452035 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.452037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.452037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.452101 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.452352 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.452353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.452354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.452503 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.452547 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.452875 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.452984 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:09:14.453219 29386 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.453299 29386 heartbeater.cc:511] Master 127.17.130.62:39477 requested a full tablet report, sending...
I20260812 01:09:14.453555 27529 ts_manager.cc:194] Re-registered known tserver with Master: b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319)
I20260812 01:09:14.453809 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.414598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.17.130.2:51971)
0812 01:09:14.414809 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.414853 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.415956 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.416346 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.416347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.416795 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.417056 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.417089 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.418375 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.418376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.419342 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.419344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.419779 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.419780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.419964 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.420645 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.420655 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.423094 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.451612 (+ 28518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.451614 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.451614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.451925 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.452198 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.452199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.452199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.452271 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.452702 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.452703 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.452751 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.453309 (+ 558us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.453444 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":13,"thread_start_us":106,"threads_started":1}
I20260812 01:09:14.454074 27529 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:41473
I20260812 01:09:14.454211 29385 heartbeater.cc:347] Connected to a master server at 127.17.130.60:45807
I20260812 01:09:14.454259 29385 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.454344 29385 heartbeater.cc:511] Master 127.17.130.60:45807 requested a full tablet report, sending...
I20260812 01:09:14.455128 28877 ts_manager.cc:194] Registered new tserver with Master: b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319)
W20260812 01:09:14.549186 29395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:14.549412 29395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:14.549437 29395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:14.551174 29395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:14.551265 29395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:14.551311 29395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:14.552944 29395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:44249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:37257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=40449
--tserver_master_addrs=127.17.130.62:39477,127.17.130.61:40421,127.17.130.60:45807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:14.553267 29395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:14.553586 29395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:14.559397 29410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.559397 29408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:14.559646 29406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:14.560104 29395 server_base.cc:1034] running on GCE node
I20260812 01:09:14.560587 29395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:14.560920 29395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:14.562119 29395 hybrid_clock.cc:648] HybridClock initialized: now 1786496954562106 us; error 32 us; skew 500 ppm
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:14.565452 29395 init.cc:377] Logged in from keytab as oryx/127.17.130.3@KRBTEST.COM (short username oryx)
I20260812 01:09:14.566004 29395 webserver.cc:460] Webserver started at http://127.17.130.3:40449/ using document root <none> and password file <none>
I20260812 01:09:14.566255 29395 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:14.566345 29395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:14.567842 29395 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:14.568652 29422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.568905 29395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:14.569018 29395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "6bac5f7f6cb94b89916ba3a13462797e"
format_stamp: "Formatted at 2026-08-12 01:09:13 on dist-test-slave-tfpf"
server_key: "f394c9763dcc61eac0419a67ab528d07"
server_key_iv: "bec8a68ef94282f171f5850195c1d63a"
server_key_version: "encryptionkey@0"
I20260812 01:09:14.569501 29395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:14.584230 29395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:14.587245 29395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:14.587451 29395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:14.587800 29395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:14.588259 29395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:14.588310 29395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.588354 29395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:14.588382 29395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:14.595798 29395 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:37257
I20260812 01:09:14.595855 29596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:37257 every 8 connection(s)
I20260812 01:09:14.596285 29395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:14.600272 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 29395
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.60@KRBTEST.COM
Aug 12 01:09:14 dist-test-slave-tfpf krb5kdc[27452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496954, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.61@KRBTEST.COM
I20260812 01:09:14.612145 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.597711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:60575 (local address 127.17.130.60:45807)
0812 01:09:14.597787 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.597789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.597799 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:14.598744 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.598745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.598759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.598793 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.599831 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.600689 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602493 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.602817 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.606905 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.606906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.606907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.606913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.609052 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.611123 (+ 2071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.611125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.611126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.611217 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.611522 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.611523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.611524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.611695 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.611742 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.611948 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.612043 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:09:14.612159 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.597179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:56081 (local address 127.17.130.62:39477)
0812 01:09:14.597245 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.597248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.597513 (+ 265us) server_negotiation.cc:407] Connection header received
0812 01:09:14.598744 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.598745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.598759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.598793 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.599831 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.600701 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602542 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.602865 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602905 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.606574 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.606575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.606575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.606584 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.609359 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.610874 (+ 1515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.610876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.610876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.610952 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.611359 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.611360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.611360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.611507 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.611565 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.611956 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.612043 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:09:14.612756 29612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.597185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:56081)
0812 01:09:14.597387 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.597433 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.598615 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.598890 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.598891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.599459 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.599693 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.599700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.600848 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.600849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.602415 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.602420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.602991 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.603088 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.603971 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.603982 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.606469 (+ 2487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.609514 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.609514 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.609515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.610372 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.611100 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.611100 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.611101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.611274 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.611600 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.611607 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.611790 (+ 183us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.612297 (+ 507us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.612512 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":55,"thread_start_us":119,"threads_started":1}
I20260812 01:09:14.613273 29613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.597511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.17.130.3:60575)
0812 01:09:14.597728 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.597738 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.598615 (+ 877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.598890 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.598891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.599450 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.599693 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.599700 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.600848 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.600849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.602408 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.602410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.602911 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.603070 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.604072 (+ 1002us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.604077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.606809 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.609352 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.609362 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.609364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.611024 (+ 1660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.611320 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.611320 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.611321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.611412 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.611792 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.611793 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.611825 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.612599 (+ 774us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.612703 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":65,"thread_start_us":72,"threads_started":1}
I20260812 01:09:14.613329 29602 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39477
I20260812 01:09:14.613651 29602 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.613916 29602 heartbeater.cc:511] Master 127.17.130.62:39477 requested a full tablet report, sending...
I20260812 01:09:14.614158 29597 heartbeater.cc:347] Connected to a master server at 127.17.130.60:45807
I20260812 01:09:14.614231 29597 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.614315 29597 heartbeater.cc:511] Master 127.17.130.60:45807 requested a full tablet report, sending...
I20260812 01:09:14.614742 28877 ts_manager.cc:194] Registered new tserver with Master: 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257)
I20260812 01:09:14.615201 27529 ts_manager.cc:194] Re-registered known tserver with Master: 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257)
I20260812 01:09:14.615792 27529 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:56081
I20260812 01:09:14.616137 29623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.601744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.17.130.61:40421)
0812 01:09:14.603079 (+ 1335us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.603081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.603090 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:14.603111 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.603112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.603127 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.603205 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.604361 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.605372 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.606431 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.606772 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.606849 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.610213 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.610214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.610215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.610223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.612270 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.613122 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.613123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.613123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.613200 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.614153 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.614154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.614154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.614365 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.614482 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.615883 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.615992 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":62,"threads_started":1}
I20260812 01:09:14.616835 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.601616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.17.130.62:39477)
0812 01:09:14.603362 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.603364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.603373 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:14.603396 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.603396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.603410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.603486 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.604299 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.605204 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.606416 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.606738 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.606808 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.608267 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.608268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.608269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.608276 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.610805 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.613290 (+ 2485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.613291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.613291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.613482 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.615212 (+ 1730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.615213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.615214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.615361 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.615455 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.616591 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.616716 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1725,"thread_start_us":63,"threads_started":1}
I20260812 01:09:14.618183 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.597500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:55899 (local address 127.17.130.61:40421)
0812 01:09:14.597559 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.597561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.597571 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:14.598697 (+ 1126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.598699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.598712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.598770 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.599818 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.600549 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602667 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.602920 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.602963 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.614438 (+ 11475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.614440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.614440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.614470 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.616469 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.617265 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.617266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.617267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.617341 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.617566 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.617567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.617568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.617718 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.617804 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.617995 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.618085 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:09:14.618211 29611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.597185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.17.130.3:55899)
0812 01:09:14.597456 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.597469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.598581 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.598849 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.598853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.599434 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:14.599692 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.599698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.601776 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.601777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.602566 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.602569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.603048 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.603048 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.603110 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.607200 (+ 4090us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:14.607204 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:14.614321 (+ 7117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:14.616703 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.616704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.616704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.617134 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.617418 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:14.617419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:14.617419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:14.617492 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:14.617812 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:14.617813 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:14.617861 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.617950 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.618089 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":25,"thread_start_us":111,"threads_started":1}
I20260812 01:09:14.618584 29607 heartbeater.cc:347] Connected to a master server at 127.17.130.61:40421
I20260812 01:09:14.618649 29607 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:14.618736 29607 heartbeater.cc:511] Master 127.17.130.61:40421 requested a full tablet report, sending...
I20260812 01:09:14.619164 28510 ts_manager.cc:194] Registered new tserver with Master: 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257)
I20260812 01:09:14.619448 27529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 01:09:14.622449 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.602117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45346 (local address 127.17.130.60:45807)
0812 01:09:14.602359 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.602361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.603353 (+ 992us) server_negotiation.cc:407] Connection header received
0812 01:09:14.603437 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.603438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.603451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.603516 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.608875 (+ 5359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.613825 (+ 4950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.615312 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.615669 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.615759 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.618852 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.618853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.618853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.618861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.620568 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.620943 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.620944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.620944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.621011 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.621580 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.621581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.621582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.621740 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.621869 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.622218 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.622323 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":58,"threads_started":1}
W20260812 01:09:14.622839 27529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:14.631706 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.627400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.17.130.3:37257)
0812 01:09:14.627638 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.627644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.627849 (+ 205us) server_negotiation.cc:407] Connection header received
0812 01:09:14.627937 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.627941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.627963 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.628176 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.628295 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.629370 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.630574 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.631325 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.631448 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.631536 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.631593 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.631629 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
I20260812 01:09:14.631698 29641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.627295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37257 (local address 127.0.0.1:33498)
0812 01:09:14.627761 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.627779 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.627861 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.628170 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.628171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.628173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.628237 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.628239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.629473 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.629474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.630460 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.630462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.631451 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.631452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.631515 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.631518 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.631561 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.631606 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":96,"threads_started":1}
I20260812 01:09:14.631834 29647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.627295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36343 (local address 127.0.0.1:50998)
0812 01:09:14.627803 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.627815 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.627875 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.628347 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.628348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.628351 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.628403 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.628405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.629798 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.629800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.630801 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.630804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.631578 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.631579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.631646 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.631651 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.631681 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.631720 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260812 01:09:14.632545 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.627517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.17.130.1:36343)
0812 01:09:14.627683 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.627687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.627942 (+ 255us) server_negotiation.cc:407] Connection header received
0812 01:09:14.627994 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.627997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.628014 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.628262 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.628496 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.629692 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.630885 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.631480 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.631962 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.632349 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.632381 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.632434 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260812 01:09:14.632730 29493 tablet_service.cc:1467] Processing CreateTablet for tablet 5268d69b15074e0496f470db497a7a1d (DEFAULT_TABLE table=test-table [id=cfd72ebef30343be9f0cfb011eb8388e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:14.633215 29493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5268d69b15074e0496f470db497a7a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:14.633311 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.628171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53136 (local address 127.17.130.2:39319)
0812 01:09:14.628575 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.628579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.628589 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:14.628629 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.628631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.628647 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.628729 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.628887 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.629985 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.631230 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.632362 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.632430 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.633145 (+ 715us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.633180 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.633224 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":116,"threads_started":1}
I20260812 01:09:14.633235 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 5268d69b15074e0496f470db497a7a1d (DEFAULT_TABLE table=test-table [id=cfd72ebef30343be9f0cfb011eb8388e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:14.633255 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.627662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39319 (local address 127.0.0.1:53136)
0812 01:09:14.627951 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.627960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.627998 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.628745 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.628746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.628748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.628805 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.628807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.630082 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.630083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.631133 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.631136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.632893 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.632894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.633020 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.633025 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.633138 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.633188 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":97,"threads_started":1}
I20260812 01:09:14.633600 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5268d69b15074e0496f470db497a7a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:14.634279 29272 tablet_service.cc:1467] Processing CreateTablet for tablet 5268d69b15074e0496f470db497a7a1d (DEFAULT_TABLE table=test-table [id=cfd72ebef30343be9f0cfb011eb8388e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:14.634657 29272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5268d69b15074e0496f470db497a7a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:14.638298 29649 tablet_bootstrap.cc:492] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Bootstrap starting.
I20260812 01:09:14.638501 29650 tablet_bootstrap.cc:492] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Bootstrap starting.
I20260812 01:09:14.639398 29649 tablet_bootstrap.cc:654] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:14.639405 29650 tablet_bootstrap.cc:654] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:14.639794 29649 log.cc:824] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:14.639794 29650 log.cc:824] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:14.640267 29654 tablet_bootstrap.cc:492] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Bootstrap starting.
I20260812 01:09:14.641381 29654 tablet_bootstrap.cc:654] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:14.641513 29649 tablet_bootstrap.cc:492] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: No bootstrap required, opened a new log
I20260812 01:09:14.641670 29649 ts_tablet_manager.cc:1397] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 01:09:14.641824 29654 log.cc:824] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:14.642118 29650 tablet_bootstrap.cc:492] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: No bootstrap required, opened a new log
I20260812 01:09:14.642246 29650 ts_tablet_manager.cc:1397] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:09:14.643455 29654 tablet_bootstrap.cc:492] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: No bootstrap required, opened a new log
I20260812 01:09:14.643553 29654 ts_tablet_manager.cc:1397] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:09:14.644356 29649 raft_consensus.cc:348] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.644546 29649 raft_consensus.cc:374] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:14.644595 29649 raft_consensus.cc:729] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f9788d8806422dadbc726dadc79bd7, State: Initialized, Role: FOLLOWER
I20260812 01:09:14.645017 29649 consensus_queue.cc:260] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.645431 29649 ts_tablet_manager.cc:1428] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:09:14.645851 29386 heartbeater.cc:503] Master 127.17.130.62:39477 was elected leader, sending a full tablet report...
I20260812 01:09:14.645606 29650 raft_consensus.cc:348] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.646112 29650 raft_consensus.cc:374] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:14.646175 29650 raft_consensus.cc:729] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bac5f7f6cb94b89916ba3a13462797e, State: Initialized, Role: FOLLOWER
I20260812 01:09:14.646322 29650 consensus_queue.cc:260] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.646320 29654 raft_consensus.cc:348] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.646472 29654 raft_consensus.cc:374] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:14.646528 29654 raft_consensus.cc:729] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d3da2d23ba4948ba554fa916cd061b, State: Initialized, Role: FOLLOWER
I20260812 01:09:14.646624 29650 ts_tablet_manager.cc:1428] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:09:14.646684 29654 consensus_queue.cc:260] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.646930 29602 heartbeater.cc:503] Master 127.17.130.62:39477 was elected leader, sending a full tablet report...
I20260812 01:09:14.647274 29654 ts_tablet_manager.cc:1428] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:09:14.647959 29158 heartbeater.cc:503] Master 127.17.130.62:39477 was elected leader, sending a full tablet report...
I20260812 01:09:14.656240 29644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.652919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.0.0.1:45362)
0812 01:09:14.653104 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.653119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.653185 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.653563 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.653564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.653566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.653648 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.653650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.654442 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.654443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.655308 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.655312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.656010 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.656011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.656069 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.656074 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.656119 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.656169 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260812 01:09:14.656369 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.653065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.17.130.60:45807)
0812 01:09:14.653258 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.653261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.653272 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:14.653386 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.653388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.653417 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.653499 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.653756 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.654334 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.655423 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.655870 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.655947 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.656193 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.656219 (+ 26us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.656265 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260812 01:09:14.656728 28892 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 01:09:14.657370 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3607476ecc8144b891de1f812c632ba7. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: NON_VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: true } } }
I20260812 01:09:14.657519 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: LEARNER
I20260812 01:09:14.657816 29631 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: attempting to promote NON_VOTER 17378047f95b45378026e0dc4e458466 to VOTER
I20260812 01:09:14.658032 29631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } }
I20260812 01:09:14.658691 28892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 LEARNER]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260812 01:09:14.658754 28531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Refusing update from remote peer 3607476ecc8144b891de1f812c632ba7: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260812 01:09:14.659029 29435 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 01:09:14.659235 29631 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260812 01:09:14.660030 29640 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 17378047f95b45378026e0dc4e458466 (127.17.130.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.660220 28531 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 17378047f95b45378026e0dc4e458466 (127.17.130.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.660286 28892 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 17378047f95b45378026e0dc4e458466 (127.17.130.60) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.661223 29640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.661324 29640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:14.661437 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.661533 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:09:14.661504 29640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.661577 29640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:14.661628 29640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.661682 29640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:14.661693 29672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3607476ecc8144b891de1f812c632ba7" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:14.661796 29672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:09:14.664808 29389 tablet.cc:2369] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:09:14.732597 29161 tablet.cc:2369] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:09:14.847218 29610 tablet.cc:2369] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:09:14.876260 29664 raft_consensus.cc:482] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:14.876415 29664 raft_consensus.cc:504] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.876832 29664 leader_election.cc:288] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257), b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319)
I20260812 01:09:14.880259 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.877429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47609 (local address 127.17.130.2:39319)
0812 01:09:14.877486 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.877488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.877539 (+ 51us) server_negotiation.cc:407] Connection header received
0812 01:09:14.877641 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.877642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.877661 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.877737 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.877906 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.878397 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.879493 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.879859 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.879962 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.879971 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.880146 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.880186 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:09:14.880273 29680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.877334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39319 (local address 127.17.130.1:47609)
0812 01:09:14.877471 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.877484 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.877557 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.877743 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.877743 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.877746 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.877815 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.877818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.878521 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.878522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.879375 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.879378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.879963 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.879964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.880035 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.880040 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.880121 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.880172 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":21,"thread_start_us":63,"threads_started":1}
I20260812 01:09:14.880348 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.877365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:59349 (local address 127.17.130.3:37257)
0812 01:09:14.877453 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.877455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.877487 (+ 32us) server_negotiation.cc:407] Connection header received
0812 01:09:14.877622 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.877624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.877640 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.877711 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:14.877897 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.878424 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.879537 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.879929 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.880029 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.880038 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.880225 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.880266 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:09:14.880614 29678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.877097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37257 (local address 127.17.130.1:59349)
0812 01:09:14.877396 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:14.877412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:14.877512 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:14.877718 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:14.877719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:14.877721 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:14.877811 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.877813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.878522 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.878524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.879418 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:14.879420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.880037 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.880038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:14.880127 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.880131 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:14.880203 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:09:14.880252 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":67,"thread_start_us":106,"threads_started":1}
I20260812 01:09:14.880688 29307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5268d69b15074e0496f470db497a7a1d" candidate_uuid: "49d3da2d23ba4948ba554fa916cd061b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f9788d8806422dadbc726dadc79bd7" is_pre_election: true
I20260812 01:09:14.880883 29307 raft_consensus.cc:2454] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49d3da2d23ba4948ba554fa916cd061b in term 0.
I20260812 01:09:14.881016 29523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5268d69b15074e0496f470db497a7a1d" candidate_uuid: "49d3da2d23ba4948ba554fa916cd061b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bac5f7f6cb94b89916ba3a13462797e" is_pre_election: true
I20260812 01:09:14.881222 29523 raft_consensus.cc:2454] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49d3da2d23ba4948ba554fa916cd061b in term 0.
I20260812 01:09:14.881240 28979 leader_election.cc:302] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d3da2d23ba4948ba554fa916cd061b, b7f9788d8806422dadbc726dadc79bd7; no voters:
I20260812 01:09:14.881376 29664 raft_consensus.cc:2781] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:14.881431 29664 raft_consensus.cc:482] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:14.881458 29664 raft_consensus.cc:3037] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:14.882265 29664 raft_consensus.cc:504] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.882431 29664 leader_election.cc:288] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [CANDIDATE]: Term 1 election: Requested vote from peers 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257), b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319)
I20260812 01:09:14.882637 29307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5268d69b15074e0496f470db497a7a1d" candidate_uuid: "49d3da2d23ba4948ba554fa916cd061b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7f9788d8806422dadbc726dadc79bd7"
I20260812 01:09:14.882678 29523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5268d69b15074e0496f470db497a7a1d" candidate_uuid: "49d3da2d23ba4948ba554fa916cd061b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bac5f7f6cb94b89916ba3a13462797e"
I20260812 01:09:14.882735 29307 raft_consensus.cc:3037] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:14.882771 29523 raft_consensus.cc:3037] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:14.883728 29523 raft_consensus.cc:2454] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d3da2d23ba4948ba554fa916cd061b in term 1.
I20260812 01:09:14.883728 29307 raft_consensus.cc:2454] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49d3da2d23ba4948ba554fa916cd061b in term 1.
I20260812 01:09:14.883953 28979 leader_election.cc:302] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49d3da2d23ba4948ba554fa916cd061b, b7f9788d8806422dadbc726dadc79bd7; no voters:
I20260812 01:09:14.884065 29664 raft_consensus.cc:2781] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:14.884202 29664 raft_consensus.cc:686] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 1 LEADER]: Becoming Leader. State: Replica: 49d3da2d23ba4948ba554fa916cd061b, State: Running, Role: LEADER
I20260812 01:09:14.884321 29664 consensus_queue.cc:237] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } }
I20260812 01:09:14.885296 27529 catalog_manager.cc:5495] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b reported cstate change: term changed from 0 to 1, leader changed from <none> to 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1). New cstate: current_term: 1 leader_uuid: "49d3da2d23ba4948ba554fa916cd061b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } health_report { overall_health: UNKNOWN } } }
I20260812 01:09:14.920948 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.918471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51004 (local address 127.17.130.1:36343)
0812 01:09:14.918567 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.918569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.918577 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:14.918629 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.918630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.918649 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.918734 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:14.918888 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.919380 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.920145 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.920351 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.920426 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.920599 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:14.920716 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:14.920812 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.920831 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.920853 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 01:09:14.923328 29523 raft_consensus.cc:1261] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Refusing update from remote peer 49d3da2d23ba4948ba554fa916cd061b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:14.923365 29307 raft_consensus.cc:1261] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Refusing update from remote peer 49d3da2d23ba4948ba554fa916cd061b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:14.923816 29682 consensus_queue.cc:1035] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:14.924014 29664 consensus_queue.cc:1035] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:14.928311 29689 mvcc.cc:205] Tried to move back new op lower bound from 7317491527362121728 to 7317491527206776832. Current Snapshot: MvccSnapshot[applied={T|T < 7317491527362121728}]
I20260812 01:09:14.928463 29688 mvcc.cc:205] Tried to move back new op lower bound from 7317491527362121728 to 7317491527206776832. Current Snapshot: MvccSnapshot[applied={T|T < 7317491527362121728}]
I20260812 01:09:14.930848 27529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:14.930984 27529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:14.932710 27529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3607476ecc8144b891de1f812c632ba7: Sending DeleteTablet for 3 replicas of tablet 5268d69b15074e0496f470db497a7a1d
I20260812 01:09:14.933100 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 5268d69b15074e0496f470db497a7a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:14 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:50998
I20260812 01:09:14.933535 29699 tablet_replica.cc:330] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: stopping tablet replica
I20260812 01:09:14.933591 29272 tablet_service.cc:1514] Processing DeleteTablet for tablet 5268d69b15074e0496f470db497a7a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:14 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:53136
I20260812 01:09:14.933656 29699 raft_consensus.cc:2229] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:14.933769 29699 raft_consensus.cc:2258] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:14.933842 29493 tablet_service.cc:1514] Processing DeleteTablet for tablet 5268d69b15074e0496f470db497a7a1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:14 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:33498
I20260812 01:09:14.934301 29699 ts_tablet_manager.cc:1905] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:14.935600 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 27467
I20260812 01:09:14.942423 29699 ts_tablet_manager.cc:1918] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:14.942557 29699 log.cc:1196] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/5268d69b15074e0496f470db497a7a1d
I20260812 01:09:14.942958 29699 ts_tablet_manager.cc:1939] T 5268d69b15074e0496f470db497a7a1d P 49d3da2d23ba4948ba554fa916cd061b: Deleting consensus metadata
I20260812 01:09:14.945657 29704 tablet_replica.cc:330] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: stopping tablet replica
I20260812 01:09:14.945844 29701 tablet_replica.cc:330] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: stopping tablet replica
I20260812 01:09:14.946022 29701 raft_consensus.cc:2229] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:14.946223 29704 raft_consensus.cc:2229] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:14.946419 29701 raft_consensus.cc:2258] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:14.946571 29704 raft_consensus.cc:2258] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:14.947315 29704 ts_tablet_manager.cc:1905] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:14.949404 29704 ts_tablet_manager.cc:1918] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:14.949540 29701 ts_tablet_manager.cc:1905] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:14.949584 29704 log.cc:1196] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/5268d69b15074e0496f470db497a7a1d
I20260812 01:09:14.949968 29704 ts_tablet_manager.cc:1939] T 5268d69b15074e0496f470db497a7a1d P 6bac5f7f6cb94b89916ba3a13462797e: Deleting consensus metadata
I20260812 01:09:14.951927 29701 ts_tablet_manager.cc:1918] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:14.952049 29701 log.cc:1196] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/5268d69b15074e0496f470db497a7a1d
I20260812 01:09:14.952346 29701 ts_tablet_manager.cc:1939] T 5268d69b15074e0496f470db497a7a1d P b7f9788d8806422dadbc726dadc79bd7: Deleting consensus metadata
W20260812 01:09:14.953212 29202 connection.cc:535] server connection from 127.0.0.1:53136 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 01:09:14.953348 28979 connection.cc:535] server connection from 127.0.0.1:50998 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 01:09:14.953610 29429 connection.cc:535] server connection from 127.0.0.1:33498 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:09:14.959784 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.935051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.17.130.60:45807)
0812 01:09:14.935109 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.935112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.945465 (+ 10353us) server_negotiation.cc:407] Connection header received
0812 01:09:14.945539 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.945540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.945557 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.945650 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.947015 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.947957 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.952345 (+ 4388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.952710 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.952762 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.955016 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.955017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.955018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.955028 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.957825 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.958279 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.958280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.958280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.958348 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.958556 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.958557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.958558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.958704 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.958818 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.959548 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.959649 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:09:14.965464 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.955995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.17.130.61:40421)
0812 01:09:14.956083 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.956086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.956280 (+ 194us) server_negotiation.cc:407] Connection header received
0812 01:09:14.956464 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.956466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.956481 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.956517 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.957669 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.958457 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.959828 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.960122 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.960176 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.962126 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.962128 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.962129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.962138 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.963933 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.964400 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.964402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.964402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.964479 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.964720 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.964721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.964722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.964866 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.964991 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.965259 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.965352 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:09:14.966269 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:14.956021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45392 (local address 127.17.130.60:45807)
0812 01:09:14.956072 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:14.956074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:14.956528 (+ 454us) server_negotiation.cc:407] Connection header received
0812 01:09:14.956660 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:14.956661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:14.956693 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:14.956729 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:14.958040 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.958837 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.960223 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:14.960534 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:14.960581 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:14.962473 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:14.962475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:14.962475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:14.962484 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:14.964237 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.964853 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.964854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.964854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.964929 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:14.965208 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:14.965209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:14.965209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:14.965355 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:14.965435 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:14.966007 (+ 572us) server_negotiation.cc:299] Negotiation successful
0812 01:09:14.966122 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:09:15.930325 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:15.930047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:49749)
0812 01:09:15.930176 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:15.930259 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260812 01:09:15.930346 29724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:15.930044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:48569)
0812 01:09:15.930213 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:15.930292 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":58,"threads_started":1}
W20260812 01:09:15.930599 29158 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:39477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:15.930612 29723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:15.929711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:37705)
0812 01:09:15.930035 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:15.930566 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":102,"threads_started":1}
W20260812 01:09:15.930616 29602 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:39477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
W20260812 01:09:15.930805 29386 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:39477 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:16.433243 29727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3607476ecc8144b891de1f812c632ba7)
I20260812 01:09:16.433403 29727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } }
I20260812 01:09:16.433707 29727 leader_election.cc:288] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477), 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61:40421)
I20260812 01:09:16.434319 29738 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3607476ecc8144b891de1f812c632ba7)
I20260812 01:09:16.434396 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.434173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36708)
0812 01:09:16.434292 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.434350 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260812 01:09:16.434520 28840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:16.434520 29738 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } }
I20260812 01:09:16.435039 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.434846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36714)
0812 01:09:16.434903 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.435008 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 01:09:16.435258 28840 leader_election.cc:334] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477): Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:16.435372 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.435049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36726)
0812 01:09:16.435293 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.435336 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":71,"threads_started":1}
W20260812 01:09:16.435485 28483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:16.435485 29738 leader_election.cc:288] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477), 17378047f95b45378026e0dc4e458466 (127.17.130.60:45807)
I20260812 01:09:16.436057 29744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.435833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36730)
0812 01:09:16.436001 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.436032 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260812 01:09:16.436753 28483 leader_election.cc:334] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477): Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:16.438097 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.434055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40421 (local address 127.0.0.1:59966)
0812 01:09:16.434198 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.434212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.434284 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.434538 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.434539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.434542 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.434623 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.434625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.435694 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.435695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.437228 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.437231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.437830 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.437831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.437897 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.437902 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.437976 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.438026 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":56,"threads_started":1}
I20260812 01:09:16.438098 29733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.434130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59966 (local address 127.17.130.61:40421)
0812 01:09:16.434259 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.434261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.434271 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:16.434381 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.434382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.434401 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.434479 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.434710 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.435584 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.437324 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.437711 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.437814 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.437829 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.437982 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.438025 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":109,"thread_start_us":55,"threads_started":1}
I20260812 01:09:16.438496 28531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17378047f95b45378026e0dc4e458466" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56987b3ad25742a2a267907febc3d2c0" is_pre_election: true
I20260812 01:09:16.438629 28531 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17378047f95b45378026e0dc4e458466 in term 1.
I20260812 01:09:16.438660 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.435091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:45807 (local address 127.0.0.1:39974)
0812 01:09:16.435737 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.435748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.435818 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.435973 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.435974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.435975 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.436020 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.436021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.436710 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.436711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.437640 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.437643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.438389 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.438390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.438458 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.438462 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.438528 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.438582 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":2,"thread_start_us":142,"threads_started":1}
I20260812 01:09:16.439040 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.435326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39974 (local address 127.17.130.60:45807)
0812 01:09:16.435663 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.435666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.435803 (+ 137us) server_negotiation.cc:407] Connection header received
0812 01:09:16.435872 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.435872 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.435882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.435958 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.436111 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.436608 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.437737 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.438124 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.438384 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.438396 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.438737 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.438959 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":95,"threads_started":1}
I20260812 01:09:16.439044 28843 leader_election.cc:302] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17378047f95b45378026e0dc4e458466, 56987b3ad25742a2a267907febc3d2c0; no voters: 3607476ecc8144b891de1f812c632ba7
I20260812 01:09:16.439229 29727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 01:09:16.439268 29727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3607476ecc8144b891de1f812c632ba7)
I20260812 01:09:16.439288 29727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:09:16.439303 28892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "56987b3ad25742a2a267907febc3d2c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "17378047f95b45378026e0dc4e458466" is_pre_election: true
I20260812 01:09:16.440085 29727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } }
I20260812 01:09:16.440268 29727 leader_election.cc:288] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [CANDIDATE]: Term 2 election: Requested vote from peers 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477), 56987b3ad25742a2a267907febc3d2c0 (127.17.130.61:40421)
I20260812 01:09:16.440308 28892 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 56987b3ad25742a2a267907febc3d2c0 in current term 2: Already voted for candidate 17378047f95b45378026e0dc4e458466 in this term.
I20260812 01:09:16.440521 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.440409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36736)
0812 01:09:16.440453 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.440493 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 01:09:16.440613 28488 leader_election.cc:302] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 56987b3ad25742a2a267907febc3d2c0; no voters: 17378047f95b45378026e0dc4e458466, 3607476ecc8144b891de1f812c632ba7
I20260812 01:09:16.440793 29738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:09:16.440924 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.440765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36740)
0812 01:09:16.440817 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.440842 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:09:16.441022 28840 leader_election.cc:334] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477): Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
I20260812 01:09:16.441707 29738 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:09:16.441864 28531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "17378047f95b45378026e0dc4e458466" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "56987b3ad25742a2a267907febc3d2c0"
I20260812 01:09:16.442821 28531 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17378047f95b45378026e0dc4e458466 in term 2.
I20260812 01:09:16.443027 28843 leader_election.cc:302] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 17378047f95b45378026e0dc4e458466, 56987b3ad25742a2a267907febc3d2c0; no voters: 3607476ecc8144b891de1f812c632ba7
I20260812 01:09:16.443132 29727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:09:16.443316 29727 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [term 2 LEADER]: Becoming Leader. State: Replica: 17378047f95b45378026e0dc4e458466, State: Running, Role: LEADER
I20260812 01:09:16.443384 29727 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } }
I20260812 01:09:16.443902 29752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17378047f95b45378026e0dc4e458466. Latest consensus state: current_term: 2 leader_uuid: "17378047f95b45378026e0dc4e458466" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:16.444095 29752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:16.444476 29754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:16.444978 29754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:16.445153 29754 catalog_manager.cc:1259] Loaded cluster ID: 39ede93edfb342fbb55be0227a32b5e6
I20260812 01:09:16.445199 29754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:16.445412 29754 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:16.445502 29754 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Loaded TSK: 0
I20260812 01:09:16.445704 29754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:09:16.457432 28874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:16.459182 28874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:16.460476 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.460326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36750)
0812 01:09:16.460397 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.460446 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 01:09:16.460528 28531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [term 2 FOLLOWER]: Refusing update from remote peer 17378047f95b45378026e0dc4e458466: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 01:09:16.460949 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.460834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.0.0.1:36766)
0812 01:09:16.460901 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.460929 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 01:09:16.460947 29752 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 01:09:16.461133 28840 consensus_peers.cc:487] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 -> Peer 3607476ecc8144b891de1f812c632ba7 (127.17.130.62:39477): Couldn't send request to peer 3607476ecc8144b891de1f812c632ba7. Status: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 01:09:16.461699 29738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17378047f95b45378026e0dc4e458466. Latest consensus state: current_term: 2 leader_uuid: "17378047f95b45378026e0dc4e458466" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:16.461799 29738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:09:16.462572 29752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "17378047f95b45378026e0dc4e458466" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:16.462693 29752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:16.463109 29756 mvcc.cc:205] Tried to move back new op lower bound from 7317491533659451392 to 7317491533592846336. Current Snapshot: MvccSnapshot[applied={T|T < 7317491533659451392}]
I20260812 01:09:16.463352 29738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "17378047f95b45378026e0dc4e458466" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "3607476ecc8144b891de1f812c632ba7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39477 } } peers { permanent_uuid: "56987b3ad25742a2a267907febc3d2c0" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "17378047f95b45378026e0dc4e458466" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 45807 } attrs { promote: false } } }
I20260812 01:09:16.463588 29738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56987b3ad25742a2a267907febc3d2c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:09:16.469491 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.465449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37257 (local address 127.0.0.1:54650)
0812 01:09:16.465525 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.465556 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.465610 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.466200 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.466201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.466205 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.466264 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.466265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.467142 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.467143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.468433 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.468436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.469213 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.469214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.469282 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.469286 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.469356 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.469403 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":36}
I20260812 01:09:16.469492 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.465829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54650 (local address 127.17.130.3:37257)
0812 01:09:16.466044 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.466046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.466054 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:16.466079 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.466080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.466092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.466195 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.466319 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.467052 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.468527 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.469144 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.469210 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.469224 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.469371 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.469413 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260812 01:09:16.469792 29493 tablet_service.cc:1467] Processing CreateTablet for tablet 707f3d2f70f64e858a85e4dcdff6e80d (DEFAULT_TABLE table=test-table [id=40b43dfefca44df68734ce28f490a39d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:16.469940 29493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707f3d2f70f64e858a85e4dcdff6e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:16.470157 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.465912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54084 (local address 127.17.130.2:39319)
0812 01:09:16.466275 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.466277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.466285 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:16.467153 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.467153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.467169 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.467252 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.467373 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.467847 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.469193 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.469794 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.469878 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.469890 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.470049 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.470090 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":136,"threads_started":1}
I20260812 01:09:16.470157 29764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.465625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39319 (local address 127.0.0.1:54084)
0812 01:09:16.465834 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.465848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.465902 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.467257 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.467258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.467261 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.467310 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.467312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.467938 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.467939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.469089 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.469092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.469891 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.469892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.469955 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.469965 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.470038 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.470087 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":5,"thread_start_us":160,"threads_started":1}
I20260812 01:09:16.470387 29272 tablet_service.cc:1467] Processing CreateTablet for tablet 707f3d2f70f64e858a85e4dcdff6e80d (DEFAULT_TABLE table=test-table [id=40b43dfefca44df68734ce28f490a39d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:16.470504 29272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707f3d2f70f64e858a85e4dcdff6e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:16.471300 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.465449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36343 (local address 127.0.0.1:60038)
0812 01:09:16.465534 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.465546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.465599 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.465992 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.465993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.465996 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.466058 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.466060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.467127 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.467129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.468433 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.468436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.471036 (+ 2600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.471037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.471105 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.471111 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.471181 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.471235 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":106}
I20260812 01:09:16.471333 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.465668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60038 (local address 127.17.130.1:36343)
0812 01:09:16.465827 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.465829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.465838 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:16.465871 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.465872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.465884 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.466131 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.466177 (+ 46us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.466964 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.469268 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.470934 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.471021 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.471036 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.471196 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.471244 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":107,"threads_started":1}
I20260812 01:09:16.471653 29774 tablet_bootstrap.cc:492] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Bootstrap starting.
I20260812 01:09:16.471807 29059 tablet_service.cc:1467] Processing CreateTablet for tablet 707f3d2f70f64e858a85e4dcdff6e80d (DEFAULT_TABLE table=test-table [id=40b43dfefca44df68734ce28f490a39d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:16.471907 29782 tablet_bootstrap.cc:492] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Bootstrap starting.
I20260812 01:09:16.471932 29059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707f3d2f70f64e858a85e4dcdff6e80d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:16.472189 29774 tablet_bootstrap.cc:654] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:16.472306 29782 tablet_bootstrap.cc:654] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:16.472841 29774 tablet_bootstrap.cc:492] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: No bootstrap required, opened a new log
I20260812 01:09:16.472878 29774 ts_tablet_manager.cc:1397] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:16.472993 29774 raft_consensus.cc:348] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.473044 29774 raft_consensus.cc:374] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:16.473124 29774 raft_consensus.cc:729] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bac5f7f6cb94b89916ba3a13462797e, State: Initialized, Role: FOLLOWER
I20260812 01:09:16.473177 29774 consensus_queue.cc:260] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.473379 29774 ts_tablet_manager.cc:1428] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:09:16.473999 29782 tablet_bootstrap.cc:492] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: No bootstrap required, opened a new log
I20260812 01:09:16.474058 29782 ts_tablet_manager.cc:1397] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:16.474072 29790 tablet_bootstrap.cc:492] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b: Bootstrap starting.
I20260812 01:09:16.474153 29597 heartbeater.cc:503] Master 127.17.130.60:45807 was elected leader, sending a full tablet report...
I20260812 01:09:16.474210 29782 raft_consensus.cc:348] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.474280 29782 raft_consensus.cc:374] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:16.474298 29782 raft_consensus.cc:729] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7f9788d8806422dadbc726dadc79bd7, State: Initialized, Role: FOLLOWER
I20260812 01:09:16.474339 29782 consensus_queue.cc:260] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.474491 29790 tablet_bootstrap.cc:654] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:16.474510 29782 ts_tablet_manager.cc:1428] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:09:16.474910 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.473576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:56539)
0812 01:09:16.474822 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.474868 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1220,"thread_start_us":95,"threads_started":1}
I20260812 01:09:16.475449 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.474658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:38803)
0812 01:09:16.475343 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.475389 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":82,"threads_started":1}
I20260812 01:09:16.475479 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.475281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:35363)
0812 01:09:16.475419 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.475446 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260812 01:09:16.475889 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.475762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:56773)
0812 01:09:16.475822 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.475852 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:09:16.475950 29790 tablet_bootstrap.cc:492] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b: No bootstrap required, opened a new log
I20260812 01:09:16.476028 29790 ts_tablet_manager.cc:1397] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:16.476145 29790 raft_consensus.cc:348] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.476223 29790 raft_consensus.cc:374] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:16.476239 29790 raft_consensus.cc:729] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d3da2d23ba4948ba554fa916cd061b, State: Initialized, Role: FOLLOWER
I20260812 01:09:16.476310 29790 consensus_queue.cc:260] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.476482 29790 ts_tablet_manager.cc:1428] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:09:16.476953 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.476670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:45925)
0812 01:09:16.476869 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.476912 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260812 01:09:16.477012 29156 heartbeater.cc:503] Master 127.17.130.60:45807 was elected leader, sending a full tablet report...
I20260812 01:09:16.477500 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.477403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:33731)
0812 01:09:16.477442 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.477475 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 01:09:16.571079 29792 raft_consensus.cc:482] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:16.571201 29792 raft_consensus.cc:504] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.571697 29792 leader_election.cc:288] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257), 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343)
I20260812 01:09:16.575667 29793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.571906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37257 (local address 127.17.130.2:54915)
0812 01:09:16.571978 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.571992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.572094 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.572311 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.572313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.572316 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.572394 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.572396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.573374 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.573375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.574446 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.574449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.575377 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.575379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.575455 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.575459 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.575523 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.575558 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:09:16.575676 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.572028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:54915 (local address 127.17.130.3:37257)
0812 01:09:16.572088 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.572091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.572103 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:09:16.572184 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.572185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.572196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.572262 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.572500 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.573282 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.574620 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.575257 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.575332 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.575375 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.575546 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.575599 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:09:16.575762 29763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.572073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:57543 (local address 127.17.130.1:36343)
0812 01:09:16.572125 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.572127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.572298 (+ 171us) server_negotiation.cc:407] Connection header received
0812 01:09:16.572341 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.572342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.572353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.572421 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:16.572733 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.573637 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.574873 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.575269 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.575391 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.575403 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.575633 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.575681 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:09:16.575876 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.571985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36343 (local address 127.17.130.2:57543)
0812 01:09:16.572200 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.572211 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:16.572270 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:16.572502 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:16.572503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:16.572506 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:16.572589 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.572591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.573744 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.573745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.574761 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:16.574765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.575444 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.575445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:16.575541 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.575545 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:16.575607 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:09:16.575649 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":134,"threads_started":1}
I20260812 01:09:16.576231 29523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707f3d2f70f64e858a85e4dcdff6e80d" candidate_uuid: "b7f9788d8806422dadbc726dadc79bd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bac5f7f6cb94b89916ba3a13462797e" is_pre_election: true
I20260812 01:09:16.576325 29523 raft_consensus.cc:2454] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7f9788d8806422dadbc726dadc79bd7 in term 0.
I20260812 01:09:16.576225 29091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707f3d2f70f64e858a85e4dcdff6e80d" candidate_uuid: "b7f9788d8806422dadbc726dadc79bd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d3da2d23ba4948ba554fa916cd061b" is_pre_election: true
I20260812 01:09:16.576416 29091 raft_consensus.cc:2454] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7f9788d8806422dadbc726dadc79bd7 in term 0.
I20260812 01:09:16.576542 29197 leader_election.cc:302] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bac5f7f6cb94b89916ba3a13462797e, b7f9788d8806422dadbc726dadc79bd7; no voters:
I20260812 01:09:16.576721 29792 raft_consensus.cc:2781] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:16.576777 29792 raft_consensus.cc:482] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:16.576802 29792 raft_consensus.cc:3037] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:16.577613 29792 raft_consensus.cc:504] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.577790 29792 leader_election.cc:288] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [CANDIDATE]: Term 1 election: Requested vote from peers 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257), 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343)
I20260812 01:09:16.578058 29523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707f3d2f70f64e858a85e4dcdff6e80d" candidate_uuid: "b7f9788d8806422dadbc726dadc79bd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bac5f7f6cb94b89916ba3a13462797e"
I20260812 01:09:16.578058 29091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707f3d2f70f64e858a85e4dcdff6e80d" candidate_uuid: "b7f9788d8806422dadbc726dadc79bd7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49d3da2d23ba4948ba554fa916cd061b"
I20260812 01:09:16.578161 29523 raft_consensus.cc:3037] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:16.578166 29091 raft_consensus.cc:3037] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:16.578961 29523 raft_consensus.cc:2454] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7f9788d8806422dadbc726dadc79bd7 in term 1.
I20260812 01:09:16.579030 29091 raft_consensus.cc:2454] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7f9788d8806422dadbc726dadc79bd7 in term 1.
I20260812 01:09:16.579180 29197 leader_election.cc:302] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bac5f7f6cb94b89916ba3a13462797e, b7f9788d8806422dadbc726dadc79bd7; no voters:
I20260812 01:09:16.579293 29792 raft_consensus.cc:2781] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:16.579491 29792 raft_consensus.cc:686] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 1 LEADER]: Becoming Leader. State: Replica: b7f9788d8806422dadbc726dadc79bd7, State: Running, Role: LEADER
I20260812 01:09:16.579615 29385 heartbeater.cc:503] Master 127.17.130.60:45807 was elected leader, sending a full tablet report...
I20260812 01:09:16.579558 29792 consensus_queue.cc:237] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } }
I20260812 01:09:16.580098 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.579756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:41235)
0812 01:09:16.579887 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.580064 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 01:09:16.580581 28877 catalog_manager.cc:5495] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "b7f9788d8806422dadbc726dadc79bd7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7f9788d8806422dadbc726dadc79bd7" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 } health_report { overall_health: UNKNOWN } } }
I20260812 01:09:16.589134 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.586643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.17.130.2:39319)
0812 01:09:16.586712 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:16.586714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:16.586722 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:16.586769 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:16.586770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:16.586789 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:16.586872 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:16.587016 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.587508 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.588267 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:16.588479 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:16.588557 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:16.588733 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:16.588862 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:16.588942 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:16.588990 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 01:09:16.589017 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:09:16.590617 29523 raft_consensus.cc:1261] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Refusing update from remote peer b7f9788d8806422dadbc726dadc79bd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:16.590811 29091 raft_consensus.cc:1261] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Refusing update from remote peer b7f9788d8806422dadbc726dadc79bd7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:16.590946 29792 consensus_queue.cc:1035] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bac5f7f6cb94b89916ba3a13462797e" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:16.591114 29802 consensus_queue.cc:1035] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49d3da2d23ba4948ba554fa916cd061b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:16.591975 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.591818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:51847)
0812 01:09:16.591878 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.591924 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:09:16.592344 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.592164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:52797)
0812 01:09:16.592283 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.592312 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 01:09:16.593665 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.593511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:39031)
0812 01:09:16.593598 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.593629 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 01:09:16.593709 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.593609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.1:44371)
0812 01:09:16.593655 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.593679 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:09:16.594293 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.594163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.3:41203)
0812 01:09:16.594226 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.594254 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:09:16.594655 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.594538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39477 (local address 127.17.130.2:53561)
0812 01:09:16.594599 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.594631 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:39477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:09:16.596498 29810 mvcc.cc:205] Tried to move back new op lower bound from 7317491534192103424 to 7317491534150574080. Current Snapshot: MvccSnapshot[applied={T|T < 7317491534192103424 or (T in {7317491534192103424})}]
I20260812 01:09:16.596748 28877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:16.597289 28877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:16.599113 28877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17378047f95b45378026e0dc4e458466: Sending DeleteTablet for 3 replicas of tablet 707f3d2f70f64e858a85e4dcdff6e80d
I20260812 01:09:16.599435 29493 tablet_service.cc:1514] Processing DeleteTablet for tablet 707f3d2f70f64e858a85e4dcdff6e80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:16 UTC) from {username='oryx', principal='oryx/127.17.130.60@KRBTEST.COM'} at 127.0.0.1:54650
I20260812 01:09:16.599447 29059 tablet_service.cc:1514] Processing DeleteTablet for tablet 707f3d2f70f64e858a85e4dcdff6e80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:16 UTC) from {username='oryx', principal='oryx/127.17.130.60@KRBTEST.COM'} at 127.0.0.1:60038
I20260812 01:09:16.599696 29824 tablet_replica.cc:330] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: stopping tablet replica
I20260812 01:09:16.599762 29824 raft_consensus.cc:2229] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:16.599812 29824 raft_consensus.cc:2258] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:16.599833 29272 tablet_service.cc:1514] Processing DeleteTablet for tablet 707f3d2f70f64e858a85e4dcdff6e80d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:16 UTC) from {username='oryx', principal='oryx/127.17.130.60@KRBTEST.COM'} at 127.0.0.1:54084
I20260812 01:09:16.600124 29825 tablet_replica.cc:330] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b: stopping tablet replica
I20260812 01:09:16.600198 29825 raft_consensus.cc:2229] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:16.600260 29825 raft_consensus.cc:2258] T 707f3d2f70f64e858a85e4dcdff6e80d P 49d3da2d23ba4948ba554fa916cd061b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:16.600271 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 28783
I20260812 01:09:16.600217 29824 ts_tablet_manager.cc:1905] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:16.600381 29826 tablet_replica.cc:330] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: stopping tablet replica
I20260812 01:09:16.600445 29826 raft_consensus.cc:2229] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:16.600553 29826 raft_consensus.cc:2258] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:16.603938 29824 ts_tablet_manager.cc:1918] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:16.604096 29824 log.cc:1196] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/707f3d2f70f64e858a85e4dcdff6e80d
I20260812 01:09:16.604161 29826 ts_tablet_manager.cc:1905] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:16.604492 29824 ts_tablet_manager.cc:1939] T 707f3d2f70f64e858a85e4dcdff6e80d P 6bac5f7f6cb94b89916ba3a13462797e: Deleting consensus metadata
I20260812 01:09:16.606575 29826 ts_tablet_manager.cc:1918] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:16.606655 29826 log.cc:1196] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/707f3d2f70f64e858a85e4dcdff6e80d
I20260812 01:09:16.606968 29826 ts_tablet_manager.cc:1939] T 707f3d2f70f64e858a85e4dcdff6e80d P b7f9788d8806422dadbc726dadc79bd7: Deleting consensus metadata
I20260812 01:09:16.617520 28839 catalog_manager.cc:4841] TS 6bac5f7f6cb94b89916ba3a13462797e (127.17.130.3:37257): tablet 707f3d2f70f64e858a85e4dcdff6e80d (table test-table [id=40b43dfefca44df68734ce28f490a39d]) successfully deleted
I20260812 01:09:16.617704 28839 catalog_manager.cc:4841] TS b7f9788d8806422dadbc726dadc79bd7 (127.17.130.2:39319): tablet 707f3d2f70f64e858a85e4dcdff6e80d (table test-table [id=40b43dfefca44df68734ce28f490a39d]) successfully deleted
I20260812 01:09:16.626896 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:16.626708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36343 (local address 127.0.0.1:60040)
0812 01:09:16.626790 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:16.626821 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:36343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 01:09:16.626991 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 29170
W20260812 01:09:16.627108 28842 catalog_manager.cc:4584] TS 49d3da2d23ba4948ba554fa916cd061b (127.17.130.1:36343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 707f3d2f70f64e858a85e4dcdff6e80d: Network error: Client connection negotiation failed: client connection to 127.17.130.1:36343: connect: Connection refused (error 111)
I20260812 01:09:16.644107 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 29395
I20260812 01:09:16.665320 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 28295
I20260812 01:09:16.685689 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 28605
2026-08-12T01:09:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5558 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:16 dist-test-slave-tfpf krb5kdc[29829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:16 dist-test-slave-tfpf krb5kdc[29829](info): set up 2 sockets
Aug 12 01:09:16 dist-test-slave-tfpf krb5kdc[29829](info): commencing operation
krb5kdc: starting...
W20260812 01:09:18.077803 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.359s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:18Z Disabled control of system clock
WARNING: no policy specified for oryx/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab.
Entry for principal oryx/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab.
I20260812 01:09:18.224334 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44805
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab
--principal=oryx/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:18.320761 29847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:18.321048 29847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:18.321117 29847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:18.322365 29847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:18.322443 29847 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 01:09:18.322468 29847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:18.322502 29847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:18.322551 29847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:18.322580 29847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:18.324280 29847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44805
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.62.keytab
--principal=oryx/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:18.324590 29847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:18.324851 29847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:09:18.330651 29847 server_base.cc:1034] running on GCE node
W20260812 01:09:18.332051 29857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.332250 29859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.332449 29854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:18.332916 29847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:18.333295 29847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:18.334514 29847 hybrid_clock.cc:648] HybridClock initialized: now 1786496958334491 us; error 66 us; skew 500 ppm
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:18.350329 29847 init.cc:377] Logged in from keytab as oryx/127.17.130.62@KRBTEST.COM (short username oryx)
I20260812 01:09:18.350955 29847 webserver.cc:460] Webserver started at http://127.17.130.62:41889/ using document root <none> and password file <none>
I20260812 01:09:18.351285 29847 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:18.351347 29847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:18.351500 29847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:18.352785 29847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "a57e5cf445b00c1899ecc595ba7ffa70"
server_key_iv: "cb6a76892c1eb027ba991757f6c2f094"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.353229 29847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "a57e5cf445b00c1899ecc595ba7ffa70"
server_key_iv: "cb6a76892c1eb027ba991757f6c2f094"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.354974 29847 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 01:09:18.356146 29874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:18.356549 29847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:18.356721 29847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "a57e5cf445b00c1899ecc595ba7ffa70"
server_key_iv: "cb6a76892c1eb027ba991757f6c2f094"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.356834 29847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:18.370774 29847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:18.373777 29847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:18.373926 29847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:18.379571 29847 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44805
I20260812 01:09:18.379657 29975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44805 every 8 connection(s)
I20260812 01:09:18.380087 29847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:18.380970 29979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:18.382266 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 29847
I20260812 01:09:18.382402 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:18.382611 17928 external_mini_cluster.cc:1428] Setting key 8f5476de6f9a2632b3c6efbf9055d05a
I20260812 01:09:18.383862 29979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Bootstrap starting.
I20260812 01:09:18.384683 29979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:18.385139 29979 log.cc:824] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:18.385923 29979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: No bootstrap required, opened a new log
I20260812 01:09:18.388382 29979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44805 } }
I20260812 01:09:18.388573 29979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:18.388630 29979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3b24c0e4bd4c2abdfed5a5b76d2589, State: Initialized, Role: FOLLOWER
I20260812 01:09:18.388769 29979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44805 } }
I20260812 01:09:18.388863 29979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:18.388906 29979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:18.388960 29979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:18.389810 29979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44805 } }
I20260812 01:09:18.389950 29979 leader_election.cc:302] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b3b24c0e4bd4c2abdfed5a5b76d2589; no voters:
I20260812 01:09:18.390172 29979 leader_election.cc:288] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:18.390208 29987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:18.390324 29987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [term 1 LEADER]: Becoming Leader. State: Replica: 3b3b24c0e4bd4c2abdfed5a5b76d2589, State: Running, Role: LEADER
I20260812 01:09:18.390424 29987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44805 } }
I20260812 01:09:18.390470 29979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:18.390792 29987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b3b24c0e4bd4c2abdfed5a5b76d2589. Latest consensus state: current_term: 1 leader_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44805 } } }
I20260812 01:09:18.390805 29989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b3b24c0e4bd4c2abdfed5a5b76d2589" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44805 } } }
I20260812 01:09:18.390991 29989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:18.391206 29987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:18.391400 29997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:18.392148 29997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:18.393949 29997 catalog_manager.cc:1347] Generated new cluster ID: b2273d7ee5af450d9524caa6f87bb640
I20260812 01:09:18.394026 29997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:18.395576 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.383327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60192 (local address 127.17.130.62:44805)
0812 01:09:18.383514 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:18.383523 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:18.383549 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:09:18.383954 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:18.383991 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:18.384057 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:18.384176 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:18.385159 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.386210 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.387389 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.387866 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.387990 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.391196 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:18.391201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:18.391202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:18.391216 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:18.393496 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.393935 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.393938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.393939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.394094 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.394319 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.394320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.394320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.394626 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:18.394782 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:18.395207 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 01:09:18.395360 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260812 01:09:18.402283 29997 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:18.402889 29997 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:18.406908 29997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Generated new TSK 0
I20260812 01:09:18.407258 29997 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab.
Entry for principal oryx/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab.
I20260812 01:09:18.453341 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44805
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:18.550653 30019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:18.550908 30019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:18.550941 30019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:18.552768 30019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:18.552848 30019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:18.552873 30019 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:09:18.552893 30019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:18.555608 30019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:18.555878 30019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:18.556195 30019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:18.562376 30024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.562400 30032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.562377 30028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:18.563105 30019 server_base.cc:1034] running on GCE node
I20260812 01:09:18.563364 30019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:18.563674 30019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:18.564881 30019 hybrid_clock.cc:648] HybridClock initialized: now 1786496958564839 us; error 62 us; skew 500 ppm
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:18.567958 30019 init.cc:377] Logged in from keytab as oryx/127.17.130.1@KRBTEST.COM (short username oryx)
I20260812 01:09:18.568526 30019 webserver.cc:460] Webserver started at http://127.17.130.1:41915/ using document root <none> and password file <none>
I20260812 01:09:18.568784 30019 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:18.568845 30019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:18.568989 30019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:18.570264 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "6a4646c6ee019fd8951340d59768d77c"
server_key_iv: "a4c0308069d1cdba4e2ddf84a7e45306"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.570698 30019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "6a4646c6ee019fd8951340d59768d77c"
server_key_iv: "a4c0308069d1cdba4e2ddf84a7e45306"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.572497 30019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:18.573657 30046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:18.574103 30019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:18.574215 30019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "6a4646c6ee019fd8951340d59768d77c"
server_key_iv: "a4c0308069d1cdba4e2ddf84a7e45306"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.574345 30019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:18.586573 30019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:18.589329 30019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:18.589776 30019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:18.590543 30019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:18.590644 30019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:18.590701 30019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:18.590734 30019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:18.600071 30019 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:33757
I20260812 01:09:18.600555 30019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:18.600590 30233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:33757 every 8 connection(s)
I20260812 01:09:18.603497 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.591068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:54015 (local address 127.17.130.62:44805)
0812 01:09:18.591181 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:18.591184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:18.591503 (+ 319us) server_negotiation.cc:407] Connection header received
0812 01:09:18.592380 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:18.592381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:18.592395 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:18.592471 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:18.593535 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.594074 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.595372 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.595665 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.595762 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.599620 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:18.599622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:18.599623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:18.599632 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:18.601341 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.601912 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.601913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.601914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.601980 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.602528 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.602528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.602529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.602672 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:18.602734 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:18.603296 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 01:09:18.603405 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 01:09:18.603853 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.591058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.17.130.1:54015)
0812 01:09:18.591293 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:18.591349 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:18.592155 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:18.592594 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:18.592599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:18.593093 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:18.593438 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.593445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.594332 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.594333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.595234 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.595238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.595776 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.595778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.595931 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.596789 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:18.596798 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:18.599444 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:18.601427 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.601432 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.601441 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.601810 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.602137 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.602140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.602141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.602404 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.602736 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:18.602741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:18.603053 (+ 312us) client_negotiation.cc:769] Sending connection context
0812 01:09:18.603221 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 01:09:18.603456 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":108,"threads_started":1}
I20260812 01:09:18.604801 30235 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44805
I20260812 01:09:18.604928 30235 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:18.605214 30235 heartbeater.cc:511] Master 127.17.130.62:44805 requested a full tablet report, sending...
I20260812 01:09:18.605939 29909 ts_manager.cc:194] Registered new tserver with Master: 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757)
I20260812 01:09:18.606778 29909 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:54015
I20260812 01:09:18.610570 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 30019
I20260812 01:09:18.610718 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:18.611016 17928 external_mini_cluster.cc:1428] Setting key 406c6cecc42bb5f2bf396affbd42fd56
I20260812 01:09:18.613791 30246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.606133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.17.130.1:58321)
0812 01:09:18.606292 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:18.606310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:18.606375 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:18.606539 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:18.606541 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:18.606737 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:18.606800 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.606802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.607805 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.607806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.608340 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.608343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.608722 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.608723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.608779 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.609594 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:18.609600 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:18.609972 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:18.612399 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.612401 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.612401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.612616 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.612855 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.612856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.612856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.612923 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.613260 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:18.613261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:18.613323 (+ 62us) client_negotiation.cc:769] Sending connection context
0812 01:09:18.613405 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:09:18.613645 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260812 01:09:18.613788 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.606196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:58321 (local address 127.17.130.62:44805)
0812 01:09:18.606245 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:18.606248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:18.606383 (+ 135us) server_negotiation.cc:407] Connection header received
0812 01:09:18.606416 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:18.606417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:18.606427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:18.606494 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:18.606921 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.607672 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.608444 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.608637 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.608708 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.610146 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:18.610149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:18.610149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:18.610162 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:18.612299 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.612710 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.612711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.612712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.612778 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.612995 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.612997 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.612997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.613184 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:18.613267 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:18.613436 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:09:18.613662 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
WARNING: no policy specified for oryx/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab.
Entry for principal oryx/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab.
I20260812 01:09:18.645419 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44805
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:18.745762 30250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:18.745998 30250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:18.746027 30250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:18.747150 30250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:18.747193 30250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:18.747217 30250 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:09:18.747236 30250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:18.748778 30250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:18.749100 30250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:18.749401 30250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:18.754974 30264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.754971 30261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:18.755124 30250 server_base.cc:1034] running on GCE node
W20260812 01:09:18.755455 30260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:18.755743 30250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:18.756052 30250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:18.757225 30250 hybrid_clock.cc:648] HybridClock initialized: now 1786496958757221 us; error 43 us; skew 500 ppm
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:18.764197 30250 init.cc:377] Logged in from keytab as oryx/127.17.130.2@KRBTEST.COM (short username oryx)
I20260812 01:09:18.764678 30250 webserver.cc:460] Webserver started at http://127.17.130.2:33087/ using document root <none> and password file <none>
I20260812 01:09:18.764907 30250 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:18.764962 30250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:18.765121 30250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:18.766320 30250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "2dfe28c089aa7d2212fa5c721fdf77e9"
server_key_iv: "3692de39adad42cba7e857718d9c9753"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.766656 30250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "2dfe28c089aa7d2212fa5c721fdf77e9"
server_key_iv: "3692de39adad42cba7e857718d9c9753"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.768154 30250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:18.769121 30280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:18.769304 30250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:18.769407 30250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "2dfe28c089aa7d2212fa5c721fdf77e9"
server_key_iv: "3692de39adad42cba7e857718d9c9753"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.769469 30250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:18.794772 30250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:18.797591 30250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:18.797741 30250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:18.798324 30250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:18.798370 30250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:18.798398 30250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:18.798413 30250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:18.810160 30250 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:33369
I20260812 01:09:18.810277 30475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:33369 every 8 connection(s)
I20260812 01:09:18.810667 30250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:18.811079 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.799031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:56463 (local address 127.17.130.62:44805)
0812 01:09:18.799118 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:18.799122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:18.799283 (+ 161us) server_negotiation.cc:407] Connection header received
0812 01:09:18.799838 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:18.799840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:18.799858 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:18.799953 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:18.800978 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.801849 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.803025 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.803368 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.803457 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.806851 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:18.806854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:18.806855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:18.806867 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:18.808880 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.809588 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.809589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.809589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.809659 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.810146 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.810147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.810148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.810323 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:18.810403 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:18.810838 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 01:09:18.810977 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:09:18.811367 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.798916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.17.130.2:56463)
0812 01:09:18.799129 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:18.799170 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:18.799695 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:18.800102 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:18.800107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:18.800522 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:18.800841 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.800848 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.802048 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.802050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.802894 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.802897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.803497 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.803498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.803677 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.804612 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:18.804625 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:18.806702 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:18.809051 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.809065 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.809068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.809491 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.809742 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.809742 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.809743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.809912 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.810438 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:18.810445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:18.810616 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 01:09:18.810758 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 01:09:18.810990 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":140,"threads_started":1}
I20260812 01:09:18.812173 30476 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44805
I20260812 01:09:18.812291 30476 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:18.812533 30476 heartbeater.cc:511] Master 127.17.130.62:44805 requested a full tablet report, sending...
I20260812 01:09:18.813051 29908 ts_manager.cc:194] Registered new tserver with Master: d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:18.813679 29908 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:56463
I20260812 01:09:18.814348 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 30250
I20260812 01:09:18.814487 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:18.814752 17928 external_mini_cluster.cc:1428] Setting key 07d402eaa380570838d0765835f55dc3
I20260812 01:09:18.821640 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.813256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:41459 (local address 127.17.130.62:44805)
0812 01:09:18.813302 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:18.813304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:18.813565 (+ 261us) server_negotiation.cc:407] Connection header received
0812 01:09:18.813593 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:18.813593 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:18.813604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:18.813684 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:18.814140 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.814751 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.815598 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.815790 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.815870 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.817189 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:18.817190 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:18.817191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:18.817199 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:18.819412 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.819898 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.819899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.819899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.819972 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:18.820380 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:18.820381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:18.820381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:18.820575 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:18.820647 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:18.821323 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 01:09:18.821422 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:09:18.821841 30485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:18.813221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.17.130.2:41459)
0812 01:09:18.813476 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:18.813490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:18.813554 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:18.813686 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:18.813686 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:18.813938 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:18.814029 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.814031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.814856 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.814858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.815443 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:18.815445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:18.815870 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:18.815871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:18.815946 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:18.816538 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:18.816543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:18.817005 (+ 462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:18.819547 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.819548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.819549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.819762 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.820200 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:18.820202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:18.820202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:18.820293 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:18.820661 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:18.820663 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:18.820734 (+ 71us) client_negotiation.cc:769] Sending connection context
0812 01:09:18.821258 (+ 524us) client_negotiation.cc:240] Negotiation successful
0812 01:09:18.821536 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for oryx/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab.
Entry for principal oryx/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab.
I20260812 01:09:18.848609 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44805
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:18.950182 30491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:18.950448 30491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:18.950479 30491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:18.951798 30491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:18.951843 30491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:18.951867 30491 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 01:09:18.951886 30491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:18.953631 30491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.17.130.62:44805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:18.953893 30491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:18.954280 30491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:18.960475 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.960454 30501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:18.960474 30504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:18.961745 30491 server_base.cc:1034] running on GCE node
I20260812 01:09:18.962059 30491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:18.962368 30491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:18.963594 30491 hybrid_clock.cc:648] HybridClock initialized: now 1786496958963566 us; error 48 us; skew 500 ppm
Aug 12 01:09:18 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:18.967887 30491 init.cc:377] Logged in from keytab as oryx/127.17.130.3@KRBTEST.COM (short username oryx)
I20260812 01:09:18.968659 30491 webserver.cc:460] Webserver started at http://127.17.130.3:33467/ using document root <none> and password file <none>
I20260812 01:09:18.968917 30491 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:18.968994 30491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:18.969183 30491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:18.970417 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "64315ab1859a4e888d316b08b8a4566f"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "1a0b65f7361712e395748b3eec6a2648"
server_key_iv: "028467a79f4fe0b73bb6f0fe6697201f"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.970853 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "64315ab1859a4e888d316b08b8a4566f"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "1a0b65f7361712e395748b3eec6a2648"
server_key_iv: "028467a79f4fe0b73bb6f0fe6697201f"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.973133 30491 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 01:09:18.974622 30519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:18.975080 30491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:18.975355 30491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "64315ab1859a4e888d316b08b8a4566f"
format_stamp: "Formatted at 2026-08-12 01:09:18 on dist-test-slave-tfpf"
server_key: "1a0b65f7361712e395748b3eec6a2648"
server_key_iv: "028467a79f4fe0b73bb6f0fe6697201f"
server_key_version: "encryptionkey@0"
I20260812 01:09:18.975513 30491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:19.014358 30491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:19.017937 30491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:19.018173 30491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:19.019291 30491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:19.019420 30491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:19.019469 30491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:19.019500 30491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:19.030923 30491 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:39645
I20260812 01:09:19.031020 30723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:39645 every 8 connection(s)
I20260812 01:09:19.031797 30491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:19.035799 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.021224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:53579 (local address 127.17.130.62:44805)
0812 01:09:19.021302 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.021305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.021316 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:19.022089 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.022092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.022116 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.022229 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.023620 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.024493 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.026560 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.026864 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.026939 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.031146 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:19.031149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:19.031149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:19.031160 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:19.034158 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.034657 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.034658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.034659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.034732 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.035010 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.035011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.035011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.035153 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:19.035230 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.035602 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.035704 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 01:09:19.035989 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.020196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.17.130.3:53579)
0812 01:09:19.020599 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.020661 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.021727 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.022366 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.022373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.023194 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:19.023467 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.023477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.024653 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.024654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.026443 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.026447 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.027031 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.027033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.027209 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.028040 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:19.028054 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:19.031003 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:19.034279 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:19.034286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:19.034289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:19.034550 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:19.034803 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:19.034804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:19.034805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:19.034931 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:19.035232 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:19.035240 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:19.035438 (+ 198us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.035556 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.035739 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":131,"threads_started":1}
I20260812 01:09:19.036809 30725 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44805
I20260812 01:09:19.036898 30725 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:19.036998 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 30491
I20260812 01:09:19.037132 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:19.037145 30725 heartbeater.cc:511] Master 127.17.130.62:44805 requested a full tablet report, sending...
I20260812 01:09:19.037375 17928 external_mini_cluster.cc:1428] Setting key 30214fdd1c3d38c9bf5ea114c6400c62
I20260812 01:09:19.037562 29909 ts_manager.cc:194] Registered new tserver with Master: 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645)
I20260812 01:09:19.038127 29909 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:53579
I20260812 01:09:19.038431 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:09:19.046375 30732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.038080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.17.130.3:44917)
0812 01:09:19.038310 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.038321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.038382 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.038600 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.038601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.038964 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:19.039088 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.039100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.040034 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.040035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.040895 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.040898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.041432 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.041433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.041481 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.042025 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:19.042030 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:19.042360 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:19.044659 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:19.044660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:19.044661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:19.044922 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:19.045246 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:19.045247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:19.045247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:19.045360 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:19.045819 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:19.045821 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:19.045914 (+ 93us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.046054 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.046200 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":176,"threads_started":1}
I20260812 01:09:19.046434 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.038140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:44917 (local address 127.17.130.62:44805)
0812 01:09:19.038196 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.038198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.038367 (+ 169us) server_negotiation.cc:407] Connection header received
0812 01:09:19.038461 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.038462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.038475 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.038507 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.039204 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.039899 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.041012 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.041324 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.041361 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.042472 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:19.042474 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:19.042474 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:19.042481 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:19.044550 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.045024 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.045025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.045026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.045138 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.045529 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.045530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.045531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.045717 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:19.045804 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.046186 (+ 382us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.046304 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496959, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:09:19.077157 29984 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:19.068252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.17.130.62:44805)
0812 01:09:19.068319 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.068321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.068360 (+ 39us) server_negotiation.cc:407] Connection header received
0812 01:09:19.068396 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.068397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.068409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.068510 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.069507 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.070241 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.071354 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.071653 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.071738 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.077024 (+ 5286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60198: BlockingRecv error: recv got EOF from 127.0.0.1:60198 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496959, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:19.088809 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.078128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.17.130.62:44805)
0812 01:09:19.078191 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.078193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.078331 (+ 138us) server_negotiation.cc:407] Connection header received
0812 01:09:19.078469 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.078471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.078483 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.078609 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.079589 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.080706 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.081765 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.082259 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.082465 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.085399 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:19.085401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:19.085401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:19.085411 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:19.087454 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.087912 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.087914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.087914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.087983 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.088199 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.088200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.088200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.088333 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:19.088455 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.088591 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.088687 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:09:19.091316 29908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:19.093858 29908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:19.103466 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.098338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39645 (local address 127.0.0.1:42190)
0812 01:09:19.098898 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.098911 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.098975 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.099299 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.099303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.099307 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.099396 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.099399 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.100777 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.100778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.102236 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.102240 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.103250 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.103251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.103311 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.103314 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.103360 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.103391 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":111,"threads_started":1}
I20260812 01:09:19.103487 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.099291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.17.130.2:33369)
0812 01:09:19.099490 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.099495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.099509 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:09:19.099557 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.099559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.099578 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.099870 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.099958 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.101105 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.102364 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.102989 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.103082 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.103173 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.103316 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.103360 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":98,"threads_started":1}
I20260812 01:09:19.103466 30760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.099021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:33369 (local address 127.0.0.1:51456)
0812 01:09:19.099181 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.099193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.099250 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.099706 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.099708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.099710 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.099780 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.099783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.101237 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.101238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.102255 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.102258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.103096 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.103097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.103191 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.103209 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.103289 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.103358 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260812 01:09:19.104310 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.097977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.17.130.1:33757)
0812 01:09:19.099859 (+ 1882us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.099864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.099874 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:19.099917 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.099919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.099934 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.100015 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.100175 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.101215 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.102829 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.103598 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.103694 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.103755 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.104183 (+ 428us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.104238 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1847,"thread_start_us":87,"threads_started":1}
I20260812 01:09:19.104095 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.097806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:33757 (local address 127.0.0.1:46156)
0812 01:09:19.097933 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.097983 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.098074 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.100020 (+ 1946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.100021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.100024 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.100079 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.100080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.101312 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.101313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.102715 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.102718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.103703 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.103705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.103899 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.103905 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.103964 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.104013 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":334,"thread_start_us":57,"threads_started":1}
I20260812 01:09:19.104408 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.098797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42190 (local address 127.17.130.3:39645)
0812 01:09:19.099024 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.099031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.099045 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:09:19.099093 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.099095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.099114 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.099205 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.099491 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.100677 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.102338 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.103034 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.103128 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.103212 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.104253 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.104297 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":135,"threads_started":1}
I20260812 01:09:19.105360 30129 tablet_service.cc:1467] Processing CreateTablet for tablet bb0d6bd50cd74f729801b63887a5f624 (DEFAULT_TABLE table=test-table [id=0639b4db818f4383827834505c98d0c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.105362 30607 tablet_service.cc:1467] Processing CreateTablet for tablet bb0d6bd50cd74f729801b63887a5f624 (DEFAULT_TABLE table=test-table [id=0639b4db818f4383827834505c98d0c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.105500 30372 tablet_service.cc:1467] Processing CreateTablet for tablet bb0d6bd50cd74f729801b63887a5f624 (DEFAULT_TABLE table=test-table [id=0639b4db818f4383827834505c98d0c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.105810 30129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0d6bd50cd74f729801b63887a5f624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.105810 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0d6bd50cd74f729801b63887a5f624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.105904 30372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb0d6bd50cd74f729801b63887a5f624. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.109812 30766 tablet_bootstrap.cc:492] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Bootstrap starting.
I20260812 01:09:19.109812 30768 tablet_bootstrap.cc:492] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Bootstrap starting.
I20260812 01:09:19.109838 30765 tablet_bootstrap.cc:492] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Bootstrap starting.
I20260812 01:09:19.110460 30766 tablet_bootstrap.cc:654] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.110527 30768 tablet_bootstrap.cc:654] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.110527 30765 tablet_bootstrap.cc:654] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.110939 30768 log.cc:824] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:19.110939 30765 log.cc:824] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:19.110971 30766 log.cc:824] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:19.111796 30765 tablet_bootstrap.cc:492] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: No bootstrap required, opened a new log
I20260812 01:09:19.111805 30768 tablet_bootstrap.cc:492] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: No bootstrap required, opened a new log
I20260812 01:09:19.111860 30765 ts_tablet_manager.cc:1397] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:19.111889 30768 ts_tablet_manager.cc:1397] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:19.113734 30765 raft_consensus.cc:348] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.113785 30768 raft_consensus.cc:348] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.113917 30765 raft_consensus.cc:374] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.113924 30768 raft_consensus.cc:374] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.113967 30765 raft_consensus.cc:729] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d320bd40bb1d449daa1a41dd1f7b8e24, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.113996 30768 raft_consensus.cc:729] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.114141 30768 consensus_queue.cc:260] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.114130 30765 consensus_queue.cc:260] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.114456 30768 ts_tablet_manager.cc:1428] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:19.114500 30725 heartbeater.cc:503] Master 127.17.130.62:44805 was elected leader, sending a full tablet report...
I20260812 01:09:19.114513 30476 heartbeater.cc:503] Master 127.17.130.62:44805 was elected leader, sending a full tablet report...
I20260812 01:09:19.114773 30765 ts_tablet_manager.cc:1428] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:19.115307 30766 tablet_bootstrap.cc:492] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: No bootstrap required, opened a new log
I20260812 01:09:19.115402 30766 ts_tablet_manager.cc:1397] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 01:09:19.116981 30766 raft_consensus.cc:348] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.117257 30766 raft_consensus.cc:374] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.117314 30766 raft_consensus.cc:729] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9395db95c96c48eb9c27a6d072bc1bf3, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.117445 30766 consensus_queue.cc:260] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.117722 30766 ts_tablet_manager.cc:1428] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:09:19.117763 30235 heartbeater.cc:503] Master 127.17.130.62:44805 was elected leader, sending a full tablet report...
W20260812 01:09:19.285800 30726 tablet.cc:2369] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:09:19.312088 30478 tablet.cc:2369] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:09:19.351635 30236 tablet.cc:2369] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:09:19.369961 30773 raft_consensus.cc:482] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:19.370117 30773 raft_consensus.cc:504] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.370556 30773 leader_election.cc:288] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:19.373870 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.371023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:44975 (local address 127.17.130.1:33757)
0812 01:09:19.371085 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.371087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.371104 (+ 17us) server_negotiation.cc:407] Connection header received
0812 01:09:19.371141 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.371142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.371150 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.371242 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.371425 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.371928 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.372946 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.373487 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.373577 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.373587 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.373755 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.373793 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:09:19.373870 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.370854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:33757 (local address 127.17.130.3:44975)
0812 01:09:19.370912 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.370951 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.371012 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.371249 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.371251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.371252 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.371341 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.371343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.372026 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.372027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.372840 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.372843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.373575 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.373576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.373667 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.373671 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.373733 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.373781 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:09:19.374575 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.370941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:42511 (local address 127.17.130.2:33369)
0812 01:09:19.371020 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.371023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.371356 (+ 333us) server_negotiation.cc:407] Connection header received
0812 01:09:19.371431 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.371433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.371445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.371546 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.371729 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.372313 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.373464 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.373918 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.374094 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.374103 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.374437 (+ 334us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.374486 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:09:19.374707 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d6bd50cd74f729801b63887a5f624" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" is_pre_election: true
I20260812 01:09:19.374866 30779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.370854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:33369 (local address 127.17.130.3:42511)
0812 01:09:19.371272 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.371281 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.371364 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.371552 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.371559 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.371561 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.371644 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.371647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.372414 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.372415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.373346 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.373350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.374251 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.374252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.374308 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.374312 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.374691 (+ 379us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.374746 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":226,"thread_start_us":118,"threads_started":1}
I20260812 01:09:19.374929 30159 raft_consensus.cc:2454] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 0.
I20260812 01:09:19.375336 30407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d6bd50cd74f729801b63887a5f624" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" is_pre_election: true
I20260812 01:09:19.375504 30407 raft_consensus.cc:2454] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 0.
I20260812 01:09:19.375640 30521 leader_election.cc:302] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, 9395db95c96c48eb9c27a6d072bc1bf3; no voters:
I20260812 01:09:19.375784 30773 raft_consensus.cc:2781] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:19.375844 30773 raft_consensus.cc:482] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:19.375869 30773 raft_consensus.cc:3037] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.376732 30773 raft_consensus.cc:504] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.376924 30773 leader_election.cc:288] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 election: Requested vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:19.377236 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d6bd50cd74f729801b63887a5f624" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
I20260812 01:09:19.377239 30407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb0d6bd50cd74f729801b63887a5f624" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
I20260812 01:09:19.377368 30159 raft_consensus.cc:3037] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.377446 30407 raft_consensus.cc:3037] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.378342 30407 raft_consensus.cc:2454] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 1.
I20260812 01:09:19.378535 30159 raft_consensus.cc:2454] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 1.
I20260812 01:09:19.378567 30524 leader_election.cc:302] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, d320bd40bb1d449daa1a41dd1f7b8e24; no voters:
I20260812 01:09:19.378686 30773 raft_consensus.cc:2781] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:19.378860 30773 raft_consensus.cc:686] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 1 LEADER]: Becoming Leader. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Running, Role: LEADER
I20260812 01:09:19.378975 30773 consensus_queue.cc:237] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } }
I20260812 01:09:19.379932 29908 catalog_manager.cc:5495] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f reported cstate change: term changed from 0 to 1, leader changed from <none> to 64315ab1859a4e888d316b08b8a4566f (127.17.130.3). New cstate: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } health_report { overall_health: UNKNOWN } } }
I20260812 01:09:19.393560 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.390646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42206 (local address 127.17.130.3:39645)
0812 01:09:19.390696 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.390698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.390708 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:19.390753 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.390753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.390781 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.390849 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:19.391101 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.391625 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.392471 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.392682 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.392752 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.393033 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.393238 (+ 205us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.393323 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.393418 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.393442 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:09:19.395793 30159 raft_consensus.cc:1261] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Refusing update from remote peer 64315ab1859a4e888d316b08b8a4566f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:19.395793 30407 raft_consensus.cc:1261] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Refusing update from remote peer 64315ab1859a4e888d316b08b8a4566f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:19.396361 30780 consensus_queue.cc:1035] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:19.396522 30773 consensus_queue.cc:1035] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:19.399542 30784 mvcc.cc:205] Tried to move back new op lower bound from 7317491545681952768 to 7317491545616957440. Current Snapshot: MvccSnapshot[applied={T|T < 7317491545681952768}]
I20260812 01:09:19.399857 30786 mvcc.cc:205] Tried to move back new op lower bound from 7317491545681952768 to 7317491545616957440. Current Snapshot: MvccSnapshot[applied={T|T < 7317491545681952768}]
I20260812 01:09:19.401845 29908 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:19.401958 29908 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:19.402803 29908 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Sending DeleteTablet for 3 replicas of tablet bb0d6bd50cd74f729801b63887a5f624
I20260812 01:09:19.403201 30372 tablet_service.cc:1514] Processing DeleteTablet for tablet bb0d6bd50cd74f729801b63887a5f624 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:19 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51456
I20260812 01:09:19.403244 30129 tablet_service.cc:1514] Processing DeleteTablet for tablet bb0d6bd50cd74f729801b63887a5f624 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:19 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:46156
I20260812 01:09:19.403455 30607 tablet_service.cc:1514] Processing DeleteTablet for tablet bb0d6bd50cd74f729801b63887a5f624 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:19 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42190
I20260812 01:09:19.403446 29908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 01:09:19.403546 30805 tablet_replica.cc:330] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: stopping tablet replica
W20260812 01:09:19.403591 29908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:19.403676 30798 tablet_replica.cc:330] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: stopping tablet replica
I20260812 01:09:19.403766 30798 raft_consensus.cc:2229] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:19.403738 30805 raft_consensus.cc:2229] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:19.403870 30798 raft_consensus.cc:2258] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:19.403875 30805 raft_consensus.cc:2258] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:19.404397 30798 ts_tablet_manager.cc:1905] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:19.405249 30129 tablet_service.cc:1467] Processing CreateTablet for tablet 3601f6888cdc44c2afd6ef3b5659592b (DEFAULT_TABLE table=test-table [id=4b919421466948658aed387ecef2bcb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.405303 30607 tablet_service.cc:1467] Processing CreateTablet for tablet 3601f6888cdc44c2afd6ef3b5659592b (DEFAULT_TABLE table=test-table [id=4b919421466948658aed387ecef2bcb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.405376 30372 tablet_service.cc:1467] Processing CreateTablet for tablet 3601f6888cdc44c2afd6ef3b5659592b (DEFAULT_TABLE table=test-table [id=4b919421466948658aed387ecef2bcb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.405397 30129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3601f6888cdc44c2afd6ef3b5659592b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.405429 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3601f6888cdc44c2afd6ef3b5659592b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.405471 30372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3601f6888cdc44c2afd6ef3b5659592b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.406766 30765 tablet_bootstrap.cc:492] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Bootstrap starting.
I20260812 01:09:19.406777 30768 tablet_bootstrap.cc:492] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Bootstrap starting.
I20260812 01:09:19.406869 30766 tablet_bootstrap.cc:492] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Bootstrap starting.
I20260812 01:09:19.407219 30765 tablet_bootstrap.cc:654] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.407235 30768 tablet_bootstrap.cc:654] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.407292 30766 tablet_bootstrap.cc:654] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.407997 30768 tablet_bootstrap.cc:492] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: No bootstrap required, opened a new log
I20260812 01:09:19.408124 30768 ts_tablet_manager.cc:1397] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:19.408259 30765 tablet_bootstrap.cc:492] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: No bootstrap required, opened a new log
I20260812 01:09:19.408315 30765 ts_tablet_manager.cc:1397] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:19.408329 30768 raft_consensus.cc:348] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.408419 30768 raft_consensus.cc:374] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.408442 30768 raft_consensus.cc:729] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.408488 30768 consensus_queue.cc:260] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.408473 30765 raft_consensus.cc:348] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.408567 30765 raft_consensus.cc:374] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.408569 30768 ts_tablet_manager.cc:1428] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:19.408594 30765 raft_consensus.cc:729] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d320bd40bb1d449daa1a41dd1f7b8e24, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.408645 30765 consensus_queue.cc:260] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.409004 30765 ts_tablet_manager.cc:1428] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:19.409076 30798 ts_tablet_manager.cc:1918] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:19.409127 30805 ts_tablet_manager.cc:1905] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:19.409153 30798 log.cc:1196] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/bb0d6bd50cd74f729801b63887a5f624
I20260812 01:09:19.409461 30798 ts_tablet_manager.cc:1939] T bb0d6bd50cd74f729801b63887a5f624 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting consensus metadata
I20260812 01:09:19.410008 29882 catalog_manager.cc:4841] TS d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369): tablet bb0d6bd50cd74f729801b63887a5f624 (table test-table [id=0639b4db818f4383827834505c98d0c1]) successfully deleted
I20260812 01:09:19.410225 30766 tablet_bootstrap.cc:492] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: No bootstrap required, opened a new log
I20260812 01:09:19.410285 30766 ts_tablet_manager.cc:1397] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 01:09:19.410434 30766 raft_consensus.cc:348] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.410493 30766 raft_consensus.cc:374] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.410512 30766 raft_consensus.cc:729] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9395db95c96c48eb9c27a6d072bc1bf3, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.410558 30766 consensus_queue.cc:260] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.410681 30766 ts_tablet_manager.cc:1428] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:19.410916 30805 ts_tablet_manager.cc:1918] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:19.411022 30805 log.cc:1196] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/bb0d6bd50cd74f729801b63887a5f624
I20260812 01:09:19.411645 30805 ts_tablet_manager.cc:1939] T bb0d6bd50cd74f729801b63887a5f624 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting consensus metadata
I20260812 01:09:19.408126 30808 tablet_replica.cc:330] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: stopping tablet replica
I20260812 01:09:19.412077 29879 catalog_manager.cc:4841] TS 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757): tablet bb0d6bd50cd74f729801b63887a5f624 (table test-table [id=0639b4db818f4383827834505c98d0c1]) successfully deleted
I20260812 01:09:19.412287 30808 raft_consensus.cc:2229] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:19.412432 30808 raft_consensus.cc:2258] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:19.412940 30808 ts_tablet_manager.cc:1905] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:19.414255 30808 ts_tablet_manager.cc:1918] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:19.414322 30808 log.cc:1196] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/bb0d6bd50cd74f729801b63887a5f624
I20260812 01:09:19.414506 30808 ts_tablet_manager.cc:1939] T bb0d6bd50cd74f729801b63887a5f624 P 64315ab1859a4e888d316b08b8a4566f: Deleting consensus metadata
I20260812 01:09:19.415350 29879 catalog_manager.cc:4841] TS 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645): tablet bb0d6bd50cd74f729801b63887a5f624 (table test-table [id=0639b4db818f4383827834505c98d0c1]) successfully deleted
I20260812 01:09:19.673597 30777 raft_consensus.cc:482] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:19.673728 30777 raft_consensus.cc:504] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.674263 30777 leader_election.cc:288] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369), 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645)
I20260812 01:09:19.677636 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.674650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:51033 (local address 127.17.130.3:39645)
0812 01:09:19.674702 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.674704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.674749 (+ 45us) server_negotiation.cc:407] Connection header received
0812 01:09:19.674842 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.674843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.674862 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.674928 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.675081 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.675561 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.676770 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.677197 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.677282 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.677319 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.677526 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.677565 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:09:19.677724 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.674494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39645 (local address 127.17.130.1:51033)
0812 01:09:19.674679 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.674692 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.674763 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.674935 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.674936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.674938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.675005 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.675007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.675659 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.675661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.676661 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.676664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.677295 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.677297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.677402 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.677407 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.677514 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.677561 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":50,"thread_start_us":68,"threads_started":1}
I20260812 01:09:19.677850 30813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.674494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:33369 (local address 127.17.130.1:47555)
0812 01:09:19.674632 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.674649 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.674747 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.674945 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.674946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.674948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:19.675018 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.675020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.675749 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.675750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.676698 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.676701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.677533 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.677534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.677621 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.677632 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.677714 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.677755 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":112,"threads_started":1}
I20260812 01:09:19.677871 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.674618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47555 (local address 127.17.130.2:33369)
0812 01:09:19.674671 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.674674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.674723 (+ 49us) server_negotiation.cc:407] Connection header received
0812 01:09:19.674845 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.674847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.674857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.674941 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:19.675110 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.675632 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.676878 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.677454 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.677533 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.677542 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.677730 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.677771 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:09:19.678167 30407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3601f6888cdc44c2afd6ef3b5659592b" candidate_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" is_pre_election: true
I20260812 01:09:19.678283 30407 raft_consensus.cc:2454] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9395db95c96c48eb9c27a6d072bc1bf3 in term 0.
I20260812 01:09:19.678438 30645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3601f6888cdc44c2afd6ef3b5659592b" candidate_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64315ab1859a4e888d316b08b8a4566f" is_pre_election: true
I20260812 01:09:19.678589 30645 raft_consensus.cc:2454] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9395db95c96c48eb9c27a6d072bc1bf3 in term 0.
I20260812 01:09:19.678831 30047 leader_election.cc:302] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, 9395db95c96c48eb9c27a6d072bc1bf3; no voters:
I20260812 01:09:19.679013 30777 raft_consensus.cc:2781] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:19.679055 30777 raft_consensus.cc:482] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:19.679078 30777 raft_consensus.cc:3037] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.680073 30777 raft_consensus.cc:504] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.680284 30777 leader_election.cc:288] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [CANDIDATE]: Term 1 election: Requested vote from peers d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369), 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645)
I20260812 01:09:19.680529 30645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3601f6888cdc44c2afd6ef3b5659592b" candidate_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64315ab1859a4e888d316b08b8a4566f"
I20260812 01:09:19.680573 30407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3601f6888cdc44c2afd6ef3b5659592b" candidate_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
I20260812 01:09:19.680629 30645 raft_consensus.cc:3037] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.680649 30407 raft_consensus.cc:3037] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.681517 30407 raft_consensus.cc:2454] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9395db95c96c48eb9c27a6d072bc1bf3 in term 1.
I20260812 01:09:19.681622 30645 raft_consensus.cc:2454] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9395db95c96c48eb9c27a6d072bc1bf3 in term 1.
I20260812 01:09:19.681787 30050 leader_election.cc:302] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9395db95c96c48eb9c27a6d072bc1bf3, d320bd40bb1d449daa1a41dd1f7b8e24; no voters:
I20260812 01:09:19.681916 30777 raft_consensus.cc:2781] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:19.682163 30777 raft_consensus.cc:686] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 LEADER]: Becoming Leader. State: Replica: 9395db95c96c48eb9c27a6d072bc1bf3, State: Running, Role: LEADER
I20260812 01:09:19.682268 30777 consensus_queue.cc:237] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.683108 29908 catalog_manager.cc:5495] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } health_report { overall_health: UNKNOWN } } }
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496958, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.62@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:19.730098 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.722209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44805 (local address 127.0.0.1:60228)
0812 01:09:19.722452 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.722466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.722542 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.722741 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.722743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.723117 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:19.723179 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.723181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.723918 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.723919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.724455 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.724456 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.724881 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.724882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.724936 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.725588 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:19.725593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:19.727053 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:19.728842 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:19.728850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:19.728852 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:19.729169 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:19.729393 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:19.729393 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:19.729393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:19.729456 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:19.729765 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:19.729772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:19.729821 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.729903 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.730016 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":119,"threads_started":1}
I20260812 01:09:19.730192 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.722284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60228 (local address 127.17.130.62:44805)
0812 01:09:19.722561 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.722562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.722572 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:19.722606 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.722606 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.722618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.722690 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.723289 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.723815 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.724555 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.724759 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.724837 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.727163 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:19.727164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:19.727164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:19.727172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:19.728748 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.729255 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.729256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.729256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.729323 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.729540 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.729541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.729541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.729665 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:19.729738 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.729999 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.730077 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":147,"threads_started":1}
I20260812 01:09:19.732583 29908 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:60228:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 01:09:19.732777 29908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:19.734886 30129 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9d2737b4de4e2d80ebd9014055c3a0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f9dcfc7fde14f56bd091f308ed2d63a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:09:19.734917 30607 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9d2737b4de4e2d80ebd9014055c3a0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f9dcfc7fde14f56bd091f308ed2d63a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:09:19.735021 30372 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9d2737b4de4e2d80ebd9014055c3a0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5f9dcfc7fde14f56bd091f308ed2d63a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 01:09:19.735090 30129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9d2737b4de4e2d80ebd9014055c3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.735103 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9d2737b4de4e2d80ebd9014055c3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.735172 30372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9d2737b4de4e2d80ebd9014055c3a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.736413 30766 tablet_bootstrap.cc:492] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: Bootstrap starting.
I20260812 01:09:19.736688 30768 tablet_bootstrap.cc:492] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: Bootstrap starting.
I20260812 01:09:19.737138 30766 tablet_bootstrap.cc:654] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.737282 30768 tablet_bootstrap.cc:654] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.737618 30765 tablet_bootstrap.cc:492] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: Bootstrap starting.
I20260812 01:09:19.737977 30766 tablet_bootstrap.cc:492] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: No bootstrap required, opened a new log
I20260812 01:09:19.738026 30766 ts_tablet_manager.cc:1397] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 01:09:19.738070 30765 tablet_bootstrap.cc:654] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.738152 30768 tablet_bootstrap.cc:492] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: No bootstrap required, opened a new log
I20260812 01:09:19.738191 30768 ts_tablet_manager.cc:1397] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:19.738188 30766 raft_consensus.cc:348] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.738266 30766 raft_consensus.cc:374] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.738288 30766 raft_consensus.cc:729] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9395db95c96c48eb9c27a6d072bc1bf3, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.738286 30768 raft_consensus.cc:348] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.738345 30768 raft_consensus.cc:374] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.738369 30768 raft_consensus.cc:729] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.738337 30766 consensus_queue.cc:260] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.738409 30768 consensus_queue.cc:260] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.738451 30766 ts_tablet_manager.cc:1428] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 01:09:19.738493 30768 ts_tablet_manager.cc:1428] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:19.738610 30777 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.738708 30777 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.738806 30773 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.738875 30773 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.738906 30765 tablet_bootstrap.cc:492] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: No bootstrap required, opened a new log
I20260812 01:09:19.738955 30765 ts_tablet_manager.cc:1397] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:19.739161 30765 raft_consensus.cc:348] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.739243 30765 raft_consensus.cc:374] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.739266 30765 raft_consensus.cc:729] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d320bd40bb1d449daa1a41dd1f7b8e24, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.739316 30765 consensus_queue.cc:260] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.739435 30765 ts_tablet_manager.cc:1428] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:19.739490 30774 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.739585 30774 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.767161 30773 raft_consensus.cc:482] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:19.767305 30773 raft_consensus.cc:504] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.767553 30773 leader_election.cc:288] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:19.767798 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9d2737b4de4e2d80ebd9014055c3a0" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" is_pre_election: true
I20260812 01:09:19.767815 30407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9d2737b4de4e2d80ebd9014055c3a0" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" is_pre_election: true
I20260812 01:09:19.767920 30407 raft_consensus.cc:2454] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 0.
I20260812 01:09:19.767932 30159 raft_consensus.cc:2454] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 0.
I20260812 01:09:19.768164 30524 leader_election.cc:302] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, d320bd40bb1d449daa1a41dd1f7b8e24; no voters:
I20260812 01:09:19.768316 30773 raft_consensus.cc:2781] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:19.768365 30773 raft_consensus.cc:482] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:19.768381 30773 raft_consensus.cc:3037] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.769119 30773 raft_consensus.cc:504] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.769273 30773 leader_election.cc:288] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 election: Requested vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:19.769479 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9d2737b4de4e2d80ebd9014055c3a0" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
I20260812 01:09:19.769472 30407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9d2737b4de4e2d80ebd9014055c3a0" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
I20260812 01:09:19.769557 30159 raft_consensus.cc:3037] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.769634 30407 raft_consensus.cc:3037] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:19.770226 30159 raft_consensus.cc:2454] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 1.
I20260812 01:09:19.770431 30521 leader_election.cc:302] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, 9395db95c96c48eb9c27a6d072bc1bf3; no voters:
I20260812 01:09:19.770455 30407 raft_consensus.cc:2454] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 1.
I20260812 01:09:19.770661 30773 raft_consensus.cc:2781] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:19.770735 30773 raft_consensus.cc:686] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [term 1 LEADER]: Becoming Leader. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Running, Role: LEADER
I20260812 01:09:19.770792 30773 consensus_queue.cc:237] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.771178 30780 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: TxnStatusTablet state changed. Reason: New leader 64315ab1859a4e888d316b08b8a4566f. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.771307 30780 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:09:19.771481 30838 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 01:09:19.771625 30838 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 01:09:19.771658 29908 catalog_manager.cc:5495] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f reported cstate change: term changed from 0 to 1, leader changed from <none> to 64315ab1859a4e888d316b08b8a4566f (127.17.130.3). New cstate: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } health_report { overall_health: HEALTHY } } }
I20260812 01:09:19.787653 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.784895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42218 (local address 127.17.130.3:39645)
0812 01:09:19.784963 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.784990 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.785000 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:19.785141 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.785143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.785156 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.785213 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:19.785374 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.785842 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.786986 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.787177 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.787242 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.787421 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.787448 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.787487 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.787573 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.787596 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:09:19.787703 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.784826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39645 (local address 127.0.0.1:42218)
0812 01:09:19.784954 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.784967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.785019 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.785234 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.785235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.785237 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:09:19.785297 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.785300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.785922 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.785923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.786883 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.786886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.787258 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.787259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.787318 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.787336 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.787508 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.787515 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.787570 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.787621 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 01:09:19.788791 30407 raft_consensus.cc:1261] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Refusing update from remote peer 64315ab1859a4e888d316b08b8a4566f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:19.788827 30159 raft_consensus.cc:1261] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Refusing update from remote peer 64315ab1859a4e888d316b08b8a4566f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:19.789048 30780 consensus_queue.cc:1035] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:19.789183 30773 consensus_queue.cc:1035] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:19.789783 30777 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: TxnStatusTablet state changed. Reason: New leader 64315ab1859a4e888d316b08b8a4566f. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.789906 30777 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.789856 30774 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: TxnStatusTablet state changed. Reason: New leader 64315ab1859a4e888d316b08b8a4566f. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.789944 30774 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.790681 30783 mvcc.cc:205] Tried to move back new op lower bound from 7317491547293147136 to 7317491547221676032. Current Snapshot: MvccSnapshot[applied={T|T < 7317491547293147136}]
I20260812 01:09:19.791088 30777 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.791121 30780 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.791217 30780 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:09:19.791213 30777 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 9395db95c96c48eb9c27a6d072bc1bf3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.791267 30780 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.791303 30780 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P 64315ab1859a4e888d316b08b8a4566f: This TxnStatusTablet replica's current role is: LEADER
I20260812 01:09:19.791671 30774 tablet_replica.cc:439] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } } }
I20260812 01:09:19.791752 30774 tablet_replica.cc:442] T 2c9d2737b4de4e2d80ebd9014055c3a0 P d320bd40bb1d449daa1a41dd1f7b8e24: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 01:09:19.795082 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.792604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.17.130.1:33757)
0812 01:09:19.792644 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.792646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.792653 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:09:19.792690 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.792691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.792716 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.792800 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:19.792908 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.793466 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.794221 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.794442 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.794523 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.794701 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.794819 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.794901 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.794948 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.794970 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:09:19.802697 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.798648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39645 (local address 127.17.130.1:52607)
0812 01:09:19.798879 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.798888 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.798942 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.799137 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.799139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.799141 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:09:19.799494 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.799496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.800586 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.800587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.801516 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.801519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.802192 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.802193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.802256 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.802274 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.802521 (+ 247us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.802524 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.802579 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.802620 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":116,"threads_started":1}
I20260812 01:09:19.802707 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.798727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:52607 (local address 127.17.130.3:39645)
0812 01:09:19.798801 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.798804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.798983 (+ 179us) server_negotiation.cc:407] Connection header received
0812 01:09:19.799014 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.799015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.799027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.799084 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:19.799590 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.800420 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.801607 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.802020 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.802109 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.802381 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.802418 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.802492 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.802606 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.802630 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:09:19.806816 30645 raft_consensus.cc:1261] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Refusing update from remote peer 9395db95c96c48eb9c27a6d072bc1bf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:19.807189 30407 raft_consensus.cc:1261] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Refusing update from remote peer 9395db95c96c48eb9c27a6d072bc1bf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:19.807621 30817 consensus_queue.cc:1035] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:19.807814 30777 consensus_queue.cc:1035] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:19.825443 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.819164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:51683 (local address 127.17.130.1:33757)
0812 01:09:19.819237 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.819240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.819463 (+ 223us) server_negotiation.cc:407] Connection header received
0812 01:09:19.820208 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.820210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.820230 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.820314 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:19.820743 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.821903 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.823618 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.823923 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.824054 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.824726 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.824807 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.824905 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.825155 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.825240 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:09:19.825496 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.819048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:33757 (local address 127.17.130.3:51683)
0812 01:09:19.819332 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.819346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.819497 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:19.820340 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:19.820341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:19.820345 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:09:19.820455 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.820458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.822110 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.822117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.823376 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:19.823389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.824078 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.824079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:19.824167 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.824209 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:09:19.824927 (+ 718us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:09:19.824932 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:19.825179 (+ 247us) client_negotiation.cc:240] Negotiation successful
0812 01:09:19.825373 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":140,"threads_started":1}
I20260812 01:09:19.843696 29906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:19.843900 29906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:19.845446 29906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Sending DeleteTablet for 3 replicas of tablet 3601f6888cdc44c2afd6ef3b5659592b
I20260812 01:09:19.846002 30129 tablet_service.cc:1514] Processing DeleteTablet for tablet 3601f6888cdc44c2afd6ef3b5659592b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:19 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:46156
I20260812 01:09:19.846191 30805 tablet_replica.cc:330] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: stopping tablet replica
I20260812 01:09:19.846306 30805 raft_consensus.cc:2229] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:19.846354 30372 tablet_service.cc:1514] Processing DeleteTablet for tablet 3601f6888cdc44c2afd6ef3b5659592b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:19 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51456
I20260812 01:09:19.846450 30805 raft_consensus.cc:2258] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:19.846520 30798 tablet_replica.cc:330] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: stopping tablet replica
I20260812 01:09:19.846580 30798 raft_consensus.cc:2229] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:19.846686 30798 raft_consensus.cc:2258] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:19.847074 30805 ts_tablet_manager.cc:1905] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:19.847791 30798 ts_tablet_manager.cc:1905] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:19.850309 30805 ts_tablet_manager.cc:1918] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:09:19.850438 30805 log.cc:1196] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/3601f6888cdc44c2afd6ef3b5659592b
I20260812 01:09:19.850607 30798 ts_tablet_manager.cc:1918] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:09:19.850765 30798 log.cc:1196] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/3601f6888cdc44c2afd6ef3b5659592b
I20260812 01:09:19.850884 30805 ts_tablet_manager.cc:1939] T 3601f6888cdc44c2afd6ef3b5659592b P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting consensus metadata
I20260812 01:09:19.851279 30798 ts_tablet_manager.cc:1939] T 3601f6888cdc44c2afd6ef3b5659592b P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting consensus metadata
I20260812 01:09:19.852303 29879 catalog_manager.cc:4841] TS 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757): tablet 3601f6888cdc44c2afd6ef3b5659592b (table test-table [id=4b919421466948658aed387ecef2bcb1]) successfully deleted
I20260812 01:09:19.852377 30607 tablet_service.cc:1514] Processing DeleteTablet for tablet 3601f6888cdc44c2afd6ef3b5659592b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:19 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42190
I20260812 01:09:19.852597 30808 tablet_replica.cc:330] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: stopping tablet replica
I20260812 01:09:19.852689 30808 raft_consensus.cc:2229] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:19.852797 30808 raft_consensus.cc:2258] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:19.852303 29882 catalog_manager.cc:4841] TS d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369): tablet 3601f6888cdc44c2afd6ef3b5659592b (table test-table [id=4b919421466948658aed387ecef2bcb1]) successfully deleted
I20260812 01:09:19.853698 30808 ts_tablet_manager.cc:1905] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:19.857833 30808 ts_tablet_manager.cc:1918] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:09:19.858003 30808 log.cc:1196] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/3601f6888cdc44c2afd6ef3b5659592b
I20260812 01:09:19.858449 30808 ts_tablet_manager.cc:1939] T 3601f6888cdc44c2afd6ef3b5659592b P 64315ab1859a4e888d316b08b8a4566f: Deleting consensus metadata
I20260812 01:09:19.859081 29879 catalog_manager.cc:4841] TS 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645): tablet 3601f6888cdc44c2afd6ef3b5659592b (table test-table [id=4b919421466948658aed387ecef2bcb1]) successfully deleted
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:09:19.904999 30829 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:19.892025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60240 (local address 127.17.130.62:44805)
0812 01:09:19.892114 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.892117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.892459 (+ 342us) server_negotiation.cc:407] Connection header received
0812 01:09:19.892595 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.892597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.892621 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.892725 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.894468 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.895497 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.897367 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.897853 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.897972 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.904886 (+ 6914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60240: BlockingRecv error: recv got EOF from 127.0.0.1:60240 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 01:09:19 dist-test-slave-tfpf krb5kdc[29829](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM
I20260812 01:09:19.927069 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.907649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60252 (local address 127.17.130.62:44805)
0812 01:09:19.907716 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.907723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.908067 (+ 344us) server_negotiation.cc:407] Connection header received
0812 01:09:19.908169 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:19.908170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:19.908190 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:19.908283 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:19.909390 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.910634 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.912086 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:19.912881 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:19.913032 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:19.920396 (+ 7364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:19.920400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:19.920400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:19.920419 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:19.924048 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.924813 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.924814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.924815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.924920 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:19.925497 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:19.925498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:19.925499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:19.926032 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:19.926426 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:19.926689 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 01:09:19.926865 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:09:19.930930 29906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:19.931299 29906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:19.934536 30129 tablet_service.cc:1467] Processing CreateTablet for tablet bc4336c55459491a887680ca1f6bc833 (DEFAULT_TABLE table=test-table [id=e35ce55cb4d740ec8fcb0d6289a4b0c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.934983 30129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc4336c55459491a887680ca1f6bc833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.934664 30372 tablet_service.cc:1467] Processing CreateTablet for tablet bc4336c55459491a887680ca1f6bc833 (DEFAULT_TABLE table=test-table [id=e35ce55cb4d740ec8fcb0d6289a4b0c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.935333 30372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc4336c55459491a887680ca1f6bc833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.937881 30766 tablet_bootstrap.cc:492] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Bootstrap starting.
I20260812 01:09:19.938607 30766 tablet_bootstrap.cc:654] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.939456 30607 tablet_service.cc:1467] Processing CreateTablet for tablet bc4336c55459491a887680ca1f6bc833 (DEFAULT_TABLE table=test-table [id=e35ce55cb4d740ec8fcb0d6289a4b0c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:19.939694 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc4336c55459491a887680ca1f6bc833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:19.940303 30766 tablet_bootstrap.cc:492] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: No bootstrap required, opened a new log
I20260812 01:09:19.940389 30766 ts_tablet_manager.cc:1397] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 01:09:19.940605 30766 raft_consensus.cc:348] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.940713 30766 raft_consensus.cc:374] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.940750 30766 raft_consensus.cc:729] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9395db95c96c48eb9c27a6d072bc1bf3, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.940845 30766 consensus_queue.cc:260] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.941006 30766 ts_tablet_manager.cc:1428] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:19.944154 30768 tablet_bootstrap.cc:492] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Bootstrap starting.
I20260812 01:09:19.944474 30765 tablet_bootstrap.cc:492] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Bootstrap starting.
I20260812 01:09:19.944589 30768 tablet_bootstrap.cc:654] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.945477 30765 tablet_bootstrap.cc:654] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:19.946069 30768 tablet_bootstrap.cc:492] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: No bootstrap required, opened a new log
I20260812 01:09:19.946135 30768 ts_tablet_manager.cc:1397] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:19.946327 30768 raft_consensus.cc:348] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.946451 30768 raft_consensus.cc:374] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.946482 30768 raft_consensus.cc:729] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.946547 30768 consensus_queue.cc:260] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.946719 30768 ts_tablet_manager.cc:1428] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:19.950366 30765 tablet_bootstrap.cc:492] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: No bootstrap required, opened a new log
I20260812 01:09:19.950515 30765 ts_tablet_manager.cc:1397] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 01:09:19.950870 30765 raft_consensus.cc:348] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.951069 30765 raft_consensus.cc:374] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:19.951124 30765 raft_consensus.cc:729] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d320bd40bb1d449daa1a41dd1f7b8e24, State: Initialized, Role: FOLLOWER
I20260812 01:09:19.951263 30765 consensus_queue.cc:260] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.953126 30765 ts_tablet_manager.cc:1428] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:09:19.997778 30774 raft_consensus.cc:482] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:19.997969 30774 raft_consensus.cc:504] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:19.998596 30774 leader_election.cc:288] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645)
I20260812 01:09:20.006714 30756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.999587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:36589 (local address 127.17.130.1:33757)
0812 01:09:19.999689 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.999691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:19.999864 (+ 173us) server_negotiation.cc:407] Connection header received
0812 01:09:20.000133 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:20.000135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:20.000194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:20.000261 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:20.000834 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.002056 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.004688 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.005765 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.005871 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.005883 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:20.006507 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 01:09:20.006591 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 01:09:20.006779 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.999462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:50311 (local address 127.17.130.3:39645)
0812 01:09:19.999546 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:19.999564 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:20.000021 (+ 457us) server_negotiation.cc:407] Connection header received
0812 01:09:20.000275 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:20.000277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:20.000287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:20.000325 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:20.001794 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.003010 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.005163 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.006069 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.006164 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.006177 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:20.006617 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 01:09:20.006689 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:09:20.007215 30893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.998958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:33757 (local address 127.17.130.2:36589)
0812 01:09:19.999695 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.999721 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:19.999906 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:20.000366 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:20.000367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:20.000369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:20.000557 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:20.000570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.002334 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.002335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:20.004333 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:20.004337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.006159 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.006160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:20.006333 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.006344 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 01:09:20.006900 (+ 556us) client_negotiation.cc:240] Negotiation successful
0812 01:09:20.006998 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":135,"threads_started":1}
I20260812 01:09:20.007872 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4336c55459491a887680ca1f6bc833" candidate_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" is_pre_election: true
I20260812 01:09:20.008030 30159 raft_consensus.cc:2454] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d320bd40bb1d449daa1a41dd1f7b8e24 in term 0.
I20260812 01:09:20.008441 30281 leader_election.cc:302] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9395db95c96c48eb9c27a6d072bc1bf3, d320bd40bb1d449daa1a41dd1f7b8e24; no voters:
I20260812 01:09:20.008751 30774 raft_consensus.cc:2781] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:20.008807 30774 raft_consensus.cc:482] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:20.008829 30774 raft_consensus.cc:3037] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:20.010296 30774 raft_consensus.cc:504] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.010601 30774 leader_election.cc:288] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [CANDIDATE]: Term 1 election: Requested vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645)
I20260812 01:09:20.011188 30890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:19.999341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39645 (local address 127.17.130.2:50311)
0812 01:09:19.999891 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:19.999902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:20.000065 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:20.000424 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:20.000425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:20.000427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:20.001576 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:20.001581 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.003168 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.003169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:20.004925 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:20.004928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.006345 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.006346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:20.006444 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.006448 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:20.010891 (+ 4443us) client_negotiation.cc:240] Negotiation successful
0812 01:09:20.011016 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":157,"threads_started":1}
I20260812 01:09:20.011161 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4336c55459491a887680ca1f6bc833" candidate_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
I20260812 01:09:20.011307 30159 raft_consensus.cc:3037] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:20.011610 30645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4336c55459491a887680ca1f6bc833" candidate_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64315ab1859a4e888d316b08b8a4566f" is_pre_election: true
I20260812 01:09:20.011721 30645 raft_consensus.cc:2454] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d320bd40bb1d449daa1a41dd1f7b8e24 in term 0.
I20260812 01:09:20.011927 30642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc4336c55459491a887680ca1f6bc833" candidate_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64315ab1859a4e888d316b08b8a4566f"
I20260812 01:09:20.012045 30642 raft_consensus.cc:3037] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:20.012846 30159 raft_consensus.cc:2454] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d320bd40bb1d449daa1a41dd1f7b8e24 in term 1.
I20260812 01:09:20.013237 30281 leader_election.cc:302] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9395db95c96c48eb9c27a6d072bc1bf3, d320bd40bb1d449daa1a41dd1f7b8e24; no voters:
I20260812 01:09:20.013490 30642 raft_consensus.cc:2454] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d320bd40bb1d449daa1a41dd1f7b8e24 in term 1.
I20260812 01:09:20.014019 30774 raft_consensus.cc:2781] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:20.014395 30774 raft_consensus.cc:686] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 LEADER]: Becoming Leader. State: Replica: d320bd40bb1d449daa1a41dd1f7b8e24, State: Running, Role: LEADER
I20260812 01:09:20.014606 30774 consensus_queue.cc:237] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.015689 29906 catalog_manager.cc:5495] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 reported cstate change: term changed from 0 to 1, leader changed from <none> to d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } health_report { overall_health: UNKNOWN } } }
I20260812 01:09:20.033402 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:20.027653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.17.130.2:33369)
0812 01:09:20.027744 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:20.027747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:20.027763 (+ 16us) server_negotiation.cc:407] Connection header received
0812 01:09:20.027879 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:20.027880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:20.027924 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:20.028086 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:20.028345 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.029629 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.031433 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.031857 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.031989 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.032407 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:20.032554 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:20.032632 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:20.033139 (+ 507us) server_negotiation.cc:299] Negotiation successful
0812 01:09:20.033198 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:09:20.035600 30159 raft_consensus.cc:1261] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Refusing update from remote peer d320bd40bb1d449daa1a41dd1f7b8e24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:20.035770 30642 raft_consensus.cc:1261] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Refusing update from remote peer d320bd40bb1d449daa1a41dd1f7b8e24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:20.036276 30894 consensus_queue.cc:1035] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:20.037974 30894 consensus_queue.cc:1035] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:20.049943 29906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:20.050216 29906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:20.051278 29906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Sending DeleteTablet for 3 replicas of tablet bc4336c55459491a887680ca1f6bc833
I20260812 01:09:20.051785 30129 tablet_service.cc:1514] Processing DeleteTablet for tablet bc4336c55459491a887680ca1f6bc833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:20 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:46156
I20260812 01:09:20.052058 30607 tablet_service.cc:1514] Processing DeleteTablet for tablet bc4336c55459491a887680ca1f6bc833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:20 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42190
I20260812 01:09:20.052090 30805 tablet_replica.cc:330] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: stopping tablet replica
I20260812 01:09:20.052248 30805 raft_consensus.cc:2229] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:20.052400 30805 raft_consensus.cc:2258] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:20.052253 30372 tablet_service.cc:1514] Processing DeleteTablet for tablet bc4336c55459491a887680ca1f6bc833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:20 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51456
I20260812 01:09:20.052634 30798 tablet_replica.cc:330] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: stopping tablet replica
I20260812 01:09:20.052729 30798 raft_consensus.cc:2229] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:20.052882 30798 raft_consensus.cc:2258] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:20.052367 30808 tablet_replica.cc:330] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: stopping tablet replica
I20260812 01:09:20.053268 30808 raft_consensus.cc:2229] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:20.053340 30808 raft_consensus.cc:2258] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:20.053906 30805 ts_tablet_manager.cc:1905] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:20.054217 30798 ts_tablet_manager.cc:1905] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:20.056990 29906 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:20.057507 29906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:20.057849 30805 ts_tablet_manager.cc:1918] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:20.057973 30805 log.cc:1196] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/bc4336c55459491a887680ca1f6bc833
I20260812 01:09:20.058349 30805 ts_tablet_manager.cc:1939] T bc4336c55459491a887680ca1f6bc833 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting consensus metadata
I20260812 01:09:20.058740 29879 catalog_manager.cc:4841] TS 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757): tablet bc4336c55459491a887680ca1f6bc833 (table test-table [id=e35ce55cb4d740ec8fcb0d6289a4b0c0]) successfully deleted
I20260812 01:09:20.060132 30808 ts_tablet_manager.cc:1905] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:20.060284 30372 tablet_service.cc:1467] Processing CreateTablet for tablet a6e51ed2f56e406f979af8325822ee12 (DEFAULT_TABLE table=test-table [id=675a4b6bf64c495aba46e82215e182c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:20.060587 30372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e51ed2f56e406f979af8325822ee12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:20.062744 30765 tablet_bootstrap.cc:492] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Bootstrap starting.
I20260812 01:09:20.063066 30808 ts_tablet_manager.cc:1918] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:20.063151 30808 log.cc:1196] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/bc4336c55459491a887680ca1f6bc833
I20260812 01:09:20.063350 30765 tablet_bootstrap.cc:654] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:20.063581 30808 ts_tablet_manager.cc:1939] T bc4336c55459491a887680ca1f6bc833 P 64315ab1859a4e888d316b08b8a4566f: Deleting consensus metadata
I20260812 01:09:20.064653 29879 catalog_manager.cc:4841] TS 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645): tablet bc4336c55459491a887680ca1f6bc833 (table test-table [id=e35ce55cb4d740ec8fcb0d6289a4b0c0]) successfully deleted
I20260812 01:09:20.064852 30607 tablet_service.cc:1467] Processing CreateTablet for tablet a6e51ed2f56e406f979af8325822ee12 (DEFAULT_TABLE table=test-table [id=675a4b6bf64c495aba46e82215e182c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:20.065217 30607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e51ed2f56e406f979af8325822ee12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:20.068065 30768 tablet_bootstrap.cc:492] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Bootstrap starting.
I20260812 01:09:20.068171 30129 tablet_service.cc:1467] Processing CreateTablet for tablet a6e51ed2f56e406f979af8325822ee12 (DEFAULT_TABLE table=test-table [id=675a4b6bf64c495aba46e82215e182c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:20.068434 30129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6e51ed2f56e406f979af8325822ee12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:20.068634 30768 tablet_bootstrap.cc:654] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:20.069804 30768 tablet_bootstrap.cc:492] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: No bootstrap required, opened a new log
I20260812 01:09:20.069875 30768 ts_tablet_manager.cc:1397] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:09:20.070044 30798 ts_tablet_manager.cc:1918] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:20.070060 30768 raft_consensus.cc:348] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.070149 30798 log.cc:1196] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/bc4336c55459491a887680ca1f6bc833
I20260812 01:09:20.070197 30768 raft_consensus.cc:374] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:20.070225 30768 raft_consensus.cc:729] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Initialized, Role: FOLLOWER
I20260812 01:09:20.070288 30768 consensus_queue.cc:260] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.070461 30768 ts_tablet_manager.cc:1428] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:20.070561 30798 ts_tablet_manager.cc:1939] T bc4336c55459491a887680ca1f6bc833 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting consensus metadata
I20260812 01:09:20.070851 30766 tablet_bootstrap.cc:492] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: Bootstrap starting.
I20260812 01:09:20.071693 30766 tablet_bootstrap.cc:654] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:20.072593 29882 catalog_manager.cc:4841] TS d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369): tablet bc4336c55459491a887680ca1f6bc833 (table test-table [id=e35ce55cb4d740ec8fcb0d6289a4b0c0]) successfully deleted
I20260812 01:09:20.074735 30765 tablet_bootstrap.cc:492] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: No bootstrap required, opened a new log
I20260812 01:09:20.074962 30765 ts_tablet_manager.cc:1397] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260812 01:09:20.075384 30765 raft_consensus.cc:348] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.075683 30765 raft_consensus.cc:374] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:20.075795 30765 raft_consensus.cc:729] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d320bd40bb1d449daa1a41dd1f7b8e24, State: Initialized, Role: FOLLOWER
I20260812 01:09:20.075873 30766 tablet_bootstrap.cc:492] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: No bootstrap required, opened a new log
I20260812 01:09:20.076073 30766 ts_tablet_manager.cc:1397] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260812 01:09:20.076021 30765 consensus_queue.cc:260] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.076465 30765 ts_tablet_manager.cc:1428] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:20.077099 30766 raft_consensus.cc:348] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.077312 30766 raft_consensus.cc:374] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:20.077431 30766 raft_consensus.cc:729] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9395db95c96c48eb9c27a6d072bc1bf3, State: Initialized, Role: FOLLOWER
I20260812 01:09:20.078037 30766 consensus_queue.cc:260] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.078608 30766 ts_tablet_manager.cc:1428] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 01:09:20.278715 30773 raft_consensus.cc:482] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:20.278884 30773 raft_consensus.cc:504] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.279280 30773 leader_election.cc:288] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:20.279628 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e51ed2f56e406f979af8325822ee12" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" is_pre_election: true
I20260812 01:09:20.279819 30159 raft_consensus.cc:2454] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 0.
I20260812 01:09:20.279754 30406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e51ed2f56e406f979af8325822ee12" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" is_pre_election: true
I20260812 01:09:20.279914 30406 raft_consensus.cc:2454] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 0.
I20260812 01:09:20.280143 30521 leader_election.cc:302] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, 9395db95c96c48eb9c27a6d072bc1bf3; no voters:
I20260812 01:09:20.280313 30773 raft_consensus.cc:2781] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:20.280355 30773 raft_consensus.cc:482] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:20.280381 30773 raft_consensus.cc:3037] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:20.281704 30773 raft_consensus.cc:504] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.281950 30773 leader_election.cc:288] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 election: Requested vote from peers 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757), d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369)
I20260812 01:09:20.282356 30406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e51ed2f56e406f979af8325822ee12" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24"
I20260812 01:09:20.282307 30159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6e51ed2f56e406f979af8325822ee12" candidate_uuid: "64315ab1859a4e888d316b08b8a4566f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9395db95c96c48eb9c27a6d072bc1bf3"
I20260812 01:09:20.282485 30406 raft_consensus.cc:3037] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:20.282503 30159 raft_consensus.cc:3037] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:20.283528 30406 raft_consensus.cc:2454] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 1.
I20260812 01:09:20.283840 30524 leader_election.cc:302] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64315ab1859a4e888d316b08b8a4566f, d320bd40bb1d449daa1a41dd1f7b8e24; no voters:
I20260812 01:09:20.283905 30159 raft_consensus.cc:2454] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64315ab1859a4e888d316b08b8a4566f in term 1.
I20260812 01:09:20.283936 30773 raft_consensus.cc:2781] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:20.283989 30773 raft_consensus.cc:686] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 1 LEADER]: Becoming Leader. State: Replica: 64315ab1859a4e888d316b08b8a4566f, State: Running, Role: LEADER
I20260812 01:09:20.284037 30773 consensus_queue.cc:237] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } }
I20260812 01:09:20.284955 29909 catalog_manager.cc:5495] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f reported cstate change: term changed from 0 to 1, leader changed from <none> to 64315ab1859a4e888d316b08b8a4566f (127.17.130.3). New cstate: current_term: 1 leader_uuid: "64315ab1859a4e888d316b08b8a4566f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64315ab1859a4e888d316b08b8a4566f" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39645 } health_report { overall_health: HEALTHY } } }
I20260812 01:09:20.323705 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:20.319712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42220 (local address 127.17.130.3:39645)
0812 01:09:20.319812 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:20.319814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:20.319833 (+ 19us) server_negotiation.cc:407] Connection header received
0812 01:09:20.319928 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:20.319929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:20.319948 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:20.320015 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:20.320184 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.320942 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.322177 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.322432 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.322547 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.323240 (+ 693us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:20.323275 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:20.323382 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:20.323583 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 01:09:20.323623 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 01:09:20.331851 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:20.328553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:48873 (local address 127.17.130.3:39645)
0812 01:09:20.328675 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:20.328678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:20.328687 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:20.328713 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:20.328713 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:20.328723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:20.328785 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:20.329015 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.329720 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.330888 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.331222 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.331311 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.331502 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:20.331537 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:20.331606 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:20.331660 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 01:09:20.331693 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 01:09:20.331871 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:20.328293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39645 (local address 127.17.130.3:48873)
0812 01:09:20.328473 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:20.328504 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:20.328581 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:20.328814 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:20.328815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:20.328817 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 01:09:20.328891 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:20.328894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.329829 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.329830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:20.330757 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:20.330760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:20.331349 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:20.331351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:20.331408 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:20.331417 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 01:09:20.331610 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 01:09:20.331612 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:09:20.331657 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:09:20.331760 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260812 01:09:20.336874 30406 raft_consensus.cc:1261] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Refusing update from remote peer 64315ab1859a4e888d316b08b8a4566f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:20.337042 30159 raft_consensus.cc:1261] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Refusing update from remote peer 64315ab1859a4e888d316b08b8a4566f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:20.337435 30847 consensus_queue.cc:1035] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d320bd40bb1d449daa1a41dd1f7b8e24" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 33369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:20.337653 30847 consensus_queue.cc:1035] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9395db95c96c48eb9c27a6d072bc1bf3" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:20.360105 29906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:20.360308 29906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:20.362360 29906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b3b24c0e4bd4c2abdfed5a5b76d2589: Sending DeleteTablet for 3 replicas of tablet a6e51ed2f56e406f979af8325822ee12
I20260812 01:09:20.362967 30372 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e51ed2f56e406f979af8325822ee12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:20 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51456
I20260812 01:09:20.362893 30129 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e51ed2f56e406f979af8325822ee12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:20 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:46156
I20260812 01:09:20.363009 30607 tablet_service.cc:1514] Processing DeleteTablet for tablet a6e51ed2f56e406f979af8325822ee12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:20 UTC) from {username='oryx', principal='oryx/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42190
I20260812 01:09:20.363154 30798 tablet_replica.cc:330] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: stopping tablet replica
I20260812 01:09:20.363233 30798 raft_consensus.cc:2229] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:20.363157 30805 tablet_replica.cc:330] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: stopping tablet replica
I20260812 01:09:20.363340 30798 raft_consensus.cc:2258] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:20.363397 30808 tablet_replica.cc:330] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: stopping tablet replica
I20260812 01:09:20.363420 30805 raft_consensus.cc:2229] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:20.363531 30805 raft_consensus.cc:2258] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:20.363565 30808 raft_consensus.cc:2229] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:09:20.363741 30808 raft_consensus.cc:2258] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:20.364168 30805 ts_tablet_manager.cc:1905] T a6e51ed2f56e406f979af8325822ee12 P 9395db95c96c48eb9c27a6d072bc1bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:20.364322 30798 ts_tablet_manager.cc:1905] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:20.364652 30808 ts_tablet_manager.cc:1905] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:20.364905 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 30019
I20260812 01:09:20.367311 30798 ts_tablet_manager.cc:1918] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:09:20.367444 30798 log.cc:1196] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/a6e51ed2f56e406f979af8325822ee12
I20260812 01:09:20.367789 30798 ts_tablet_manager.cc:1939] T a6e51ed2f56e406f979af8325822ee12 P d320bd40bb1d449daa1a41dd1f7b8e24: Deleting consensus metadata
I20260812 01:09:20.367985 30808 ts_tablet_manager.cc:1918] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 01:09:20.368106 30808 log.cc:1196] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/a6e51ed2f56e406f979af8325822ee12
I20260812 01:09:20.368592 30808 ts_tablet_manager.cc:1939] T a6e51ed2f56e406f979af8325822ee12 P 64315ab1859a4e888d316b08b8a4566f: Deleting consensus metadata
I20260812 01:09:20.382305 29882 catalog_manager.cc:4841] TS d320bd40bb1d449daa1a41dd1f7b8e24 (127.17.130.2:33369): tablet a6e51ed2f56e406f979af8325822ee12 (table test-table [id=675a4b6bf64c495aba46e82215e182c5]) successfully deleted
I20260812 01:09:20.389806 29879 catalog_manager.cc:4841] TS 64315ab1859a4e888d316b08b8a4566f (127.17.130.3:39645): tablet a6e51ed2f56e406f979af8325822ee12 (table test-table [id=675a4b6bf64c495aba46e82215e182c5]) successfully deleted
W20260812 01:09:20.406414 29879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:33757 (error 108)
I20260812 01:09:20.406740 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 30250
I20260812 01:09:20.415452 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:20.414856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:33757 (local address 127.0.0.1:46184)
0812 01:09:20.415238 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:20.415363 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:33757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":115,"threads_started":1}
W20260812 01:09:20.415673 29879 catalog_manager.cc:4584] TS 9395db95c96c48eb9c27a6d072bc1bf3 (127.17.130.1:33757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6e51ed2f56e406f979af8325822ee12: Network error: Client connection negotiation failed: client connection to 127.17.130.1:33757: connect: Connection refused (error 111)
I20260812 01:09:20.432718 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 30491
I20260812 01:09:20.459169 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 29847
2026-08-12T01:09:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3799 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:20 dist-test-slave-tfpf krb5kdc[30928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:20 dist-test-slave-tfpf krb5kdc[30928](info): set up 2 sockets
Aug 12 01:09:20 dist-test-slave-tfpf krb5kdc[30928](info): commencing operation
krb5kdc: starting...
W20260812 01:09:21.984655 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.466s user 0.003s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:22 dist-test-slave-tfpf krb5kdc[30928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:22.128803 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:45895
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:45895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:22.218286 30946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:22.218571 30946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:22.218613 30946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:22.219879 30946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:22.219965 30946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:22.219990 30946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:22.220011 30946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:22.220045 30946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:22.221735 30946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:45895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:22.222061 30946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:22.222332 30946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:22.228042 30955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:22.228042 30959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:22.228072 30956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:22.228430 30946 server_base.cc:1034] running on GCE node
I20260812 01:09:22.228811 30946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:22.229205 30946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:22.230381 30946 hybrid_clock.cc:648] HybridClock initialized: now 1786496962230358 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T01:09:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1741 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:22 dist-test-slave-tfpf krb5kdc[30967](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:22 dist-test-slave-tfpf krb5kdc[30967](info): set up 2 sockets
Aug 12 01:09:22 dist-test-slave-tfpf krb5kdc[30967](info): commencing operation
krb5kdc: starting...
W20260812 01:09:23.569654 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:23 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:23.717521 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34419
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:34419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:23.815189 30992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:23.815483 30992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:23.815526 30992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:23.817338 30992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:23.817425 30992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:23.817451 30992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:23.817471 30992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:23.817492 30992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:23.819273 30992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:34419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:23.819610 30992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:23.819878 30992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:23.825416 31003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:23.825416 31001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:23.825830 30992 server_base.cc:1034] running on GCE node
W20260812 01:09:23.825418 30999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:23.826285 30992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:23.826637 30992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:23.827857 30992 hybrid_clock.cc:648] HybridClock initialized: now 1786496963827840 us; error 32 us; skew 500 ppm
Aug 12 01:09:23 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496963, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:23.836833 30992 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:23.837671 30992 webserver.cc:460] Webserver started at http://127.17.130.62:43537/ using document root <none> and password file <none>
I20260812 01:09:23.837954 30992 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:23.838042 30992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:23.838207 30992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:23.839406 30992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "1be3b1fbade3479286617b17cf2673bc"
format_stamp: "Formatted at 2026-08-12 01:09:23 on dist-test-slave-tfpf"
server_key: "10670bf815c9add0ef9153e40bea6e5a"
server_key_iv: "da6fa2165be104fb39fe9933da855267"
server_key_version: "encryptionkey@0"
I20260812 01:09:23.839819 30992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "1be3b1fbade3479286617b17cf2673bc"
format_stamp: "Formatted at 2026-08-12 01:09:23 on dist-test-slave-tfpf"
server_key: "10670bf815c9add0ef9153e40bea6e5a"
server_key_iv: "da6fa2165be104fb39fe9933da855267"
server_key_version: "encryptionkey@0"
I20260812 01:09:23.841526 30992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:09:23.842550 31012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:23.842861 30992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:23.843125 30992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "1be3b1fbade3479286617b17cf2673bc"
format_stamp: "Formatted at 2026-08-12 01:09:23 on dist-test-slave-tfpf"
server_key: "10670bf815c9add0ef9153e40bea6e5a"
server_key_iv: "da6fa2165be104fb39fe9933da855267"
server_key_version: "encryptionkey@0"
I20260812 01:09:23.843300 30992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:23.870796 30992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:23.873413 30992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:23.873618 30992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:23.881259 30992 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:34419
I20260812 01:09:23.881441 31096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:34419 every 8 connection(s)
I20260812 01:09:23.881832 30992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:23.882927 31097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:23.885742 31097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Bootstrap starting.
I20260812 01:09:23.886392 31097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:23.886648 31097 log.cc:824] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:23.887422 31097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: No bootstrap required, opened a new log
I20260812 01:09:23.888727 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 30992
I20260812 01:09:23.888825 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:23.889055 17928 external_mini_cluster.cc:1428] Setting key 3a4d21d23fe387fac5bb79ce21c04470
I20260812 01:09:23.889398 31097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:23.889586 31097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:23.890033 31097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be3b1fbade3479286617b17cf2673bc, State: Initialized, Role: FOLLOWER
I20260812 01:09:23.890204 31097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:23.890304 31097 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:23.890343 31097 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:23.890389 31097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:23.891557 31097 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:23.891685 31097 leader_election.cc:302] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be3b1fbade3479286617b17cf2673bc; no voters:
I20260812 01:09:23.891870 31097 leader_election.cc:288] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:23.891916 31105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:23.892140 31105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 LEADER]: Becoming Leader. State: Replica: 1be3b1fbade3479286617b17cf2673bc, State: Running, Role: LEADER
I20260812 01:09:23.892132 31097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:23.892307 31105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:23.892772 31106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1be3b1fbade3479286617b17cf2673bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } } }
I20260812 01:09:23.892863 31106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:23.893138 31107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be3b1fbade3479286617b17cf2673bc. Latest consensus state: current_term: 1 leader_uuid: "1be3b1fbade3479286617b17cf2673bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } } }
I20260812 01:09:23.893267 31107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:23.893437 31115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:23.894054 31115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:23.897123 31115 catalog_manager.cc:1347] Generated new cluster ID: e17f1006aead405a98c3ef106e084b3c
I20260812 01:09:23.897197 31115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:09:23 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:23.902427 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:23.889714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33238 (local address 127.17.130.62:34419)
0812 01:09:23.890003 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:23.890014 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:23.890062 (+ 48us) server_negotiation.cc:407] Connection header received
0812 01:09:23.890199 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:23.890207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:23.890268 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:23.890466 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:23.891188 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:23.892616 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:23.893958 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:23.894672 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:23.894806 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:23.897977 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:23.897981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:23.897982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:23.897998 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:23.899763 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:23.900245 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:23.900248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:23.900249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:23.900657 (+ 408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:23.901120 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:23.901121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:23.901122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:23.901523 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:23.901782 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:23.901882 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 01:09:23.902133 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":107,"threads_started":1}
I20260812 01:09:23.924431 31115 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:23.925168 31115 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:23.930303 31115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Generated new TSK 0
I20260812 01:09:23.930529 31115 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:09:23.986554 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34419
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:24.088632 31134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:24.088881 31134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:24.088909 31134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:24.090368 31134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:24.090449 31134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:24.090481 31134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:24.092149 31134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:24.092484 31134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:24.092790 31134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:24.099149 31141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.099143 31143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.099184 31139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:24.099604 31134 server_base.cc:1034] running on GCE node
I20260812 01:09:24.099992 31134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:24.100332 31134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:24.101547 31134 hybrid_clock.cc:648] HybridClock initialized: now 1786496964101528 us; error 38 us; skew 500 ppm
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:24.104569 31134 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:09:24.105206 31134 webserver.cc:460] Webserver started at http://127.17.130.1:35895/ using document root <none> and password file <none>
I20260812 01:09:24.105516 31134 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:24.105572 31134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:24.105783 31134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:24.107085 31134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "be331d4c9c8a45e09d527a9a7004b742"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "6adfb46790c01041029b213b3fc22c60"
server_key_iv: "af21fa84790dcd9b9fb9d6053619a681"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.107508 31134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "be331d4c9c8a45e09d527a9a7004b742"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "6adfb46790c01041029b213b3fc22c60"
server_key_iv: "af21fa84790dcd9b9fb9d6053619a681"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.109226 31134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 01:09:24.110100 31153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.110304 31134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:24.110424 31134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "be331d4c9c8a45e09d527a9a7004b742"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "6adfb46790c01041029b213b3fc22c60"
server_key_iv: "af21fa84790dcd9b9fb9d6053619a681"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.110508 31134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:24.127569 31134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:24.130239 31134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:24.130448 31134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:24.130759 31134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:24.131117 31134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:24.131177 31134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.131222 31134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:24.131253 31134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.138208 31134 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:35219
I20260812 01:09:24.138243 31348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:35219 every 8 connection(s)
I20260812 01:09:24.138718 31134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:24.145228 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 31134
I20260812 01:09:24.145351 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:24.145620 17928 external_mini_cluster.cc:1428] Setting key 40f59e4dbaea3a6b28b10b1115e8064a
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:24.150838 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.139697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:57893 (local address 127.17.130.62:34419)
0812 01:09:24.139772 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:24.139774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:24.139976 (+ 202us) server_negotiation.cc:407] Connection header received
0812 01:09:24.140465 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:24.140466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:24.140478 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:24.140547 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:24.141663 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.142205 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.143353 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.143668 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.143814 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.146918 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:24.146920 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:24.146920 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:24.146973 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:24.149172 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:24.149688 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:24.149690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:24.149690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:24.149759 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:24.150048 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:24.150049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:24.150049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:24.150190 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:24.150277 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:24.150613 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 01:09:24.150734 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:09:24.151046 31357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.139597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:57893)
0812 01:09:24.139773 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:24.139820 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:24.140329 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:24.140606 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:24.140609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:24.141325 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:24.141548 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.141554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.142321 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.142322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.143198 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.143201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.143856 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.143857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.144001 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.144744 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:24.144753 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:24.146775 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:24.149289 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:24.149299 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:24.149302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:24.149581 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:24.149844 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:24.149845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:24.149845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:24.149972 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:24.150265 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:24.150273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:24.150436 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 01:09:24.150558 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 01:09:24.150765 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260812 01:09:24.151787 31350 heartbeater.cc:347] Connected to a master server at 127.17.130.62:34419
I20260812 01:09:24.151927 31350 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:24.152184 31350 heartbeater.cc:511] Master 127.17.130.62:34419 requested a full tablet report, sending...
I20260812 01:09:24.152905 31042 ts_manager.cc:194] Registered new tserver with Master: be331d4c9c8a45e09d527a9a7004b742 (127.17.130.1:35219)
I20260812 01:09:24.153589 31042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:57893
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:09:24.182307 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34419
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:24.279731 31362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:24.280052 31362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:24.280079 31362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:24.281297 31362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:24.281337 31362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:24.281363 31362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:24.282799 31362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:24.283179 31362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:24.283428 31362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:24.289388 31377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.289379 31375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:24.289417 31362 server_base.cc:1034] running on GCE node
W20260812 01:09:24.289387 31374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:24.289907 31362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:24.290256 31362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:24.291463 31362 hybrid_clock.cc:648] HybridClock initialized: now 1786496964291449 us; error 38 us; skew 500 ppm
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:24.294724 31362 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:09:24.295321 31362 webserver.cc:460] Webserver started at http://127.17.130.2:39267/ using document root <none> and password file <none>
I20260812 01:09:24.295570 31362 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:24.295643 31362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:24.295800 31362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:24.297202 31362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "ea58426c9e1f46adb964b36ea7b94b25"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "fd9abae8d678da53118a2066456a58b1"
server_key_iv: "99243cdcb498218374e4677b3d12274f"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.297600 31362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "ea58426c9e1f46adb964b36ea7b94b25"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "fd9abae8d678da53118a2066456a58b1"
server_key_iv: "99243cdcb498218374e4677b3d12274f"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.299098 31362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:24.299917 31387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.300119 31362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 01:09:24.300235 31362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "ea58426c9e1f46adb964b36ea7b94b25"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "fd9abae8d678da53118a2066456a58b1"
server_key_iv: "99243cdcb498218374e4677b3d12274f"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.300297 31362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:24.307753 31362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:24.310171 31362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:24.310346 31362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:24.310583 31362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:24.310870 31362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:24.310902 31362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.310925 31362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:24.310937 31362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.317840 31362 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:45839
I20260812 01:09:24.317886 31589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:45839 every 8 connection(s)
I20260812 01:09:24.318367 31362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:24.320631 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 31362
I20260812 01:09:24.320731 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:24.320960 17928 external_mini_cluster.cc:1428] Setting key d7b090c2fc52f0793ba00a4c6f40729b
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:24.330011 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.319223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58537 (local address 127.17.130.62:34419)
0812 01:09:24.319328 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:24.319331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:24.319533 (+ 202us) server_negotiation.cc:407] Connection header received
0812 01:09:24.320119 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:24.320121 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:24.320134 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:24.320183 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:24.321630 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.322251 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.323281 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.323495 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.323575 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.326358 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:24.326359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:24.326360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:24.326368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:24.328184 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:24.328705 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:24.328707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:24.328707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:24.328777 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:24.329229 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:24.329230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:24.329231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:24.329396 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:24.329489 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:24.329790 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 01:09:24.329897 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 01:09:24.330051 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.319142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:58537)
0812 01:09:24.319388 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:24.319437 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:24.319995 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:24.320266 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:24.320270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:24.321224 (+ 954us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:24.321504 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.321512 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.322352 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.322354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.323177 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.323180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.323585 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.323586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.323726 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.324472 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:24.324484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:24.326245 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:24.328301 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:24.328317 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:24.328319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:24.328613 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:24.328898 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:24.328899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:24.328900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:24.329024 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:24.329485 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:24.329491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:24.329627 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 01:09:24.329721 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 01:09:24.329868 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
I20260812 01:09:24.331044 31591 heartbeater.cc:347] Connected to a master server at 127.17.130.62:34419
I20260812 01:09:24.331174 31591 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:24.331403 31591 heartbeater.cc:511] Master 127.17.130.62:34419 requested a full tablet report, sending...
I20260812 01:09:24.331874 31042 ts_manager.cc:194] Registered new tserver with Master: ea58426c9e1f46adb964b36ea7b94b25 (127.17.130.2:45839)
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:09:24.332325 31042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:58537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:09:24.357484 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34419
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:24.452261 31600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:24.452549 31600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:24.452593 31600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:24.453949 31600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:24.454030 31600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:24.454079 31600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:24.455683 31600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:24.455994 31600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:24.456257 31600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:09:24.462280 31600 server_base.cc:1034] running on GCE node
W20260812 01:09:24.462239 31609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.462247 31610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.462249 31613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:24.462766 31600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:24.463073 31600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:24.464247 31600 hybrid_clock.cc:648] HybridClock initialized: now 1786496964464188 us; error 72 us; skew 500 ppm
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:24.467248 31600 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:09:24.467720 31600 webserver.cc:460] Webserver started at http://127.17.130.3:46361/ using document root <none> and password file <none>
I20260812 01:09:24.467976 31600 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:24.468048 31600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:24.468210 31600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:24.469436 31600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "892beaf7efaa470cacd538e3d8108cfb"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "22add290bc6f411333c7ab184e56b2a3"
server_key_iv: "56b9e880f6df15bed427fc5c9b3c7573"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.469815 31600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "892beaf7efaa470cacd538e3d8108cfb"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "22add290bc6f411333c7ab184e56b2a3"
server_key_iv: "56b9e880f6df15bed427fc5c9b3c7573"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.471485 31600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:24.472420 31623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.472671 31600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:24.472793 31600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "892beaf7efaa470cacd538e3d8108cfb"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "22add290bc6f411333c7ab184e56b2a3"
server_key_iv: "56b9e880f6df15bed427fc5c9b3c7573"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.472882 31600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:24.486225 31600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:24.488729 31600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:24.488931 31600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:24.489344 31600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:24.489728 31600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:24.489789 31600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.489835 31600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:24.489866 31600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.497452 31600 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:42361
I20260812 01:09:24.497504 31798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:42361 every 8 connection(s)
I20260812 01:09:24.497884 31600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:24.506076 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 31600
I20260812 01:09:24.506182 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:24.506399 17928 external_mini_cluster.cc:1428] Setting key 0887f8ba96456b3919ed8132647c9889
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:24.510759 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.499253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:38633 (local address 127.17.130.62:34419)
0812 01:09:24.499335 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:24.499338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:24.499741 (+ 403us) server_negotiation.cc:407] Connection header received
0812 01:09:24.500575 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:24.500576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:24.500592 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:24.500665 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:24.502019 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.502556 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.504060 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.504279 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.504353 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.507648 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:24.507649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:24.507649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:24.507657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:24.509222 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:24.509684 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:24.509685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:24.509685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:24.509750 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:24.510094 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:24.510095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:24.510096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:24.510233 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:24.510311 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:24.510610 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 01:09:24.510690 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:09:24.510917 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.499197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:38633)
0812 01:09:24.499523 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:24.499575 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:24.500400 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:24.500823 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:24.500828 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:24.501608 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:24.501840 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.501846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.502784 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.502787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.503920 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.503933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.504395 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.504396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.504525 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.505250 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:24.505268 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:24.507526 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:24.509307 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:24.509315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:24.509317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:24.509592 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:24.509859 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:24.509860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:24.509860 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:24.509995 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:24.510314 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:24.510319 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:24.510449 (+ 130us) client_negotiation.cc:769] Sending connection context
0812 01:09:24.510543 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 01:09:24.510693 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":213,"threads_started":1}
I20260812 01:09:24.511456 31799 heartbeater.cc:347] Connected to a master server at 127.17.130.62:34419
I20260812 01:09:24.511582 31799 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:24.511785 31799 heartbeater.cc:511] Master 127.17.130.62:34419 requested a full tablet report, sending...
I20260812 01:09:24.512238 31042 ts_manager.cc:194] Registered new tserver with Master: 892beaf7efaa470cacd538e3d8108cfb (127.17.130.3:42361)
I20260812 01:09:24.512636 31042 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:38633
I20260812 01:09:24.519500 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab.
Entry for principal oryx/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab.
WARNING: no policy specified for oryx/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab.
Entry for principal oryx/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab.
WARNING: no policy specified for oryx/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab.
Entry for principal oryx/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab.
I20260812 01:09:24.603318 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 31134
I20260812 01:09:24.620064 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 31362
I20260812 01:09:24.636013 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 31600
I20260812 01:09:24.668231 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 30992
I20260812 01:09:24.680064 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34419
--webserver_interface=127.17.130.62
--webserver_port=43537
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:34419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:24.776754 31813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:24.777009 31813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:24.777036 31813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:24.778522 31813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:24.778584 31813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:24.778610 31813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:24.778623 31813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:24.778636 31813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:24.780485 31813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:34419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=43537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:24.780735 31813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:24.780959 31813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:24.786248 31824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.786278 31822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.786278 31828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:24.786792 31813 server_base.cc:1034] running on GCE node
I20260812 01:09:24.787127 31813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:24.787475 31813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:24.788673 31813 hybrid_clock.cc:648] HybridClock initialized: now 1786496964788663 us; error 58 us; skew 500 ppm
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:24.791399 31813 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:24.791859 31813 webserver.cc:460] Webserver started at http://127.17.130.62:43537/ using document root <none> and password file <none>
I20260812 01:09:24.792137 31813 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:24.792235 31813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:24.793782 31813 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:09:24.794615 31835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.794852 31813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:24.794986 31813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "1be3b1fbade3479286617b17cf2673bc"
format_stamp: "Formatted at 2026-08-12 01:09:23 on dist-test-slave-tfpf"
server_key: "10670bf815c9add0ef9153e40bea6e5a"
server_key_iv: "da6fa2165be104fb39fe9933da855267"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.795398 31813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:24.823535 31813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:24.826334 31813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:24.826510 31813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:24.830842 31813 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:34419
I20260812 01:09:24.830936 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:34419 every 8 connection(s)
I20260812 01:09:24.831326 31813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:24.833012 31921 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:09:24.833755 31921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Bootstrap starting.
I20260812 01:09:24.837281 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 31813
I20260812 01:09:24.837721 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:35219
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:34419
--webserver_port=35895
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:09:24.844537 31921 log.cc:824] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:24.846491 31921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:09:24.846815 31921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Bootstrap complete.
I20260812 01:09:24.849355 31921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:24.849638 31921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be3b1fbade3479286617b17cf2673bc, State: Initialized, Role: FOLLOWER
I20260812 01:09:24.849865 31921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:24.849980 31921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:24.850041 31921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:24.850096 31921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:09:24.851186 31921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:24.851321 31921 leader_election.cc:302] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be3b1fbade3479286617b17cf2673bc; no voters:
I20260812 01:09:24.851559 31921 leader_election.cc:288] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:09:24.851600 31927 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:09:24.851732 31927 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [term 2 LEADER]: Becoming Leader. State: Replica: 1be3b1fbade3479286617b17cf2673bc, State: Running, Role: LEADER
I20260812 01:09:24.851874 31921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:24.851905 31927 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } }
I20260812 01:09:24.852424 31929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be3b1fbade3479286617b17cf2673bc. Latest consensus state: current_term: 2 leader_uuid: "1be3b1fbade3479286617b17cf2673bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } } }
I20260812 01:09:24.852438 31928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1be3b1fbade3479286617b17cf2673bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be3b1fbade3479286617b17cf2673bc" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34419 } } }
I20260812 01:09:24.852576 31928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:24.852527 31929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:24.852849 31938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:24.853773 31938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:24.853998 31938 catalog_manager.cc:1259] Loaded cluster ID: e17f1006aead405a98c3ef106e084b3c
I20260812 01:09:24.854202 31938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:24.855038 31938 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:24.855259 31938 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1be3b1fbade3479286617b17cf2673bc: Loaded TSK: 0
I20260812 01:09:24.855497 31938 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:09:24.933393 31924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:24.933619 31924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:24.933647 31924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:24.934880 31924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:24.934928 31924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:24.934957 31924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:24.936486 31924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.1.keytab
--principal=oryx/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:35219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=35895
--tserver_master_addrs=127.17.130.62:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:24.936780 31924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:24.937086 31924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:24.942399 31962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.942586 31963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:24.942368 31965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:24.942775 31924 server_base.cc:1034] running on GCE node
I20260812 01:09:24.942991 31924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:24.943251 31924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:24.944454 31924 hybrid_clock.cc:648] HybridClock initialized: now 1786496964944437 us; error 33 us; skew 500 ppm
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496964, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:24.947484 31924 init.cc:377] Logged in from keytab as oryx/127.17.130.1@KRBTEST.COM (short username oryx)
I20260812 01:09:24.948052 31924 webserver.cc:460] Webserver started at http://127.17.130.1:35895/ using document root <none> and password file <none>
I20260812 01:09:24.948333 31924 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:24.948423 31924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:24.949944 31924 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:24.950618 31982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.950832 31924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:24.950953 31924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "be331d4c9c8a45e09d527a9a7004b742"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "6adfb46790c01041029b213b3fc22c60"
server_key_iv: "af21fa84790dcd9b9fb9d6053619a681"
server_key_version: "encryptionkey@0"
I20260812 01:09:24.951488 31924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:24.961719 31924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:24.964119 31924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:24.964295 31924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:24.964547 31924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:24.964910 31924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:24.964974 31924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.965018 31924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:24.965044 31924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:24.972806 31924 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:35219
I20260812 01:09:24.973439 31924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:24.975910 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 31924
I20260812 01:09:24.976264 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:45839
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:34419
--webserver_port=39267
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:09:24.977259 32178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:35219 every 8 connection(s)
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:24.987903 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.974564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:44009)
0812 01:09:24.977443 (+ 2879us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:24.977488 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:24.978239 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:24.978795 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:24.978799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:24.979581 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:24.979887 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.979894 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.981797 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.981798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.983010 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.983013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.983949 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.983950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.984092 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.984780 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:24.984790 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:24.987686 (+ 2896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2834,"thread_start_us":100,"threads_started":1}
W20260812 01:09:24.988103 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:24.974601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:44009 (local address 127.17.130.62:34419)
0812 01:09:24.974817 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:24.974825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:24.977619 (+ 2794us) server_negotiation.cc:407] Connection header received
0812 01:09:24.978490 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:24.978498 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:24.978559 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:24.978722 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:24.980034 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.981675 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.983122 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.983830 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.983977 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.987872 (+ 3895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:44009: BlockingRecv error: recv got EOF from 127.17.130.1:44009 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":98,"threads_started":1}
W20260812 01:09:24.988596 32182 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:34419 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.130.62:34419: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:24.994289 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.988855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:46435)
0812 01:09:24.988967 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:24.988979 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:24.989036 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:24.989224 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:24.989225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:24.989377 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:24.989432 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.989433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.990330 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.990331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.991156 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.991158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.991651 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.991652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.991723 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.992218 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:24.992224 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:24.994214 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260812 01:09:24.994619 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:24.988920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:46435 (local address 127.17.130.62:34419)
0812 01:09:24.988984 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:24.988986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:24.989018 (+ 32us) server_negotiation.cc:407] Connection header received
0812 01:09:24.989135 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:24.989137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:24.989151 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:24.989203 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:24.989521 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.990242 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.991242 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.991560 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.991647 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.994322 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:46435: BlockingRecv error: recv got EOF from 127.17.130.1:46435 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:24 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:25.000319 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:24.994756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:40045)
0812 01:09:24.995034 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:24.995043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:24.995091 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:24.995235 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:24.995236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:24.995380 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:24.995415 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.995417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.996341 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.996341 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.997170 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:24.997172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.997636 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.997638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:24.997710 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:24.998216 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:24.998224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.000257 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263}
W20260812 01:09:25.000420 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:24.994822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:40045 (local address 127.17.130.62:34419)
0812 01:09:24.995005 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:24.995007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:24.995098 (+ 91us) server_negotiation.cc:407] Connection header received
0812 01:09:24.995151 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:24.995152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:24.995163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:24.995215 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:24.995489 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.996258 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.997261 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:24.997556 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:24.997619 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.000356 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:40045: BlockingRecv error: recv got EOF from 127.17.130.1:40045 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
W20260812 01:09:25.000675 32182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:09:25.067561 32191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:25.067781 32191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:25.067806 32191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:25.069000 32191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:25.069051 32191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:25.069160 32191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:25.070696 32191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.2.keytab
--principal=oryx/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:45839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=39267
--tserver_master_addrs=127.17.130.62:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:25.070957 32191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:25.071221 32191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:25.076818 32200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:25.076946 32198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:25.076992 32191 server_base.cc:1034] running on GCE node
W20260812 01:09:25.077091 32204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:25.077365 32191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:25.077656 32191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:25.078855 32191 hybrid_clock.cc:648] HybridClock initialized: now 1786496965078836 us; error 42 us; skew 500 ppm
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496965, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:25.081871 32191 init.cc:377] Logged in from keytab as oryx/127.17.130.2@KRBTEST.COM (short username oryx)
I20260812 01:09:25.082365 32191 webserver.cc:460] Webserver started at http://127.17.130.2:39267/ using document root <none> and password file <none>
I20260812 01:09:25.082613 32191 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:25.082700 32191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:25.084177 32191 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:25.084966 32217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:25.085211 32191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:25.085340 32191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "ea58426c9e1f46adb964b36ea7b94b25"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "fd9abae8d678da53118a2066456a58b1"
server_key_iv: "99243cdcb498218374e4677b3d12274f"
server_key_version: "encryptionkey@0"
I20260812 01:09:25.085739 32191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:25.101575 32191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:25.104030 32191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:25.104208 32191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:25.104477 32191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:25.104804 32191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:25.104859 32191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:25.104912 32191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:25.104943 32191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:25.111593 32191 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:45839
I20260812 01:09:25.111637 32410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:45839 every 8 connection(s)
I20260812 01:09:25.112043 32191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:25.113114 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 32191
I20260812 01:09:25.113402 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:42361
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:34419
--webserver_port=46361
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:09:25.121100 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:25.112842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58361 (local address 127.17.130.62:34419)
0812 01:09:25.112921 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.112923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.113287 (+ 364us) server_negotiation.cc:407] Connection header received
0812 01:09:25.113890 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.113892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.113911 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.113992 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.115322 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.115903 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.117217 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.117412 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.117493 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.121016 (+ 3523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:58361: BlockingRecv error: recv got EOF from 127.17.130.2:58361 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 01:09:25.121181 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.112847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:58361)
0812 01:09:25.113139 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:25.113189 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:25.113761 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:25.114075 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:25.114079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:25.114868 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:25.115154 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.115170 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.116014 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.116016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.117053 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.117085 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.117520 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.117522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.117683 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.118300 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:25.118313 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.120939 (+ 2626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
W20260812 01:09:25.121541 32411 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:34419 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.130.62:34419: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:25.126154 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.121828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:48573)
0812 01:09:25.121873 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:25.121883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:25.121943 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:25.122106 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:25.122107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:25.122254 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:25.122297 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.122299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.122922 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.122923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.123426 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.123428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.123818 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.123819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.123887 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.124347 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:25.124352 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.126066 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 01:09:25.126209 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:25.121896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:48573 (local address 127.17.130.62:34419)
0812 01:09:25.121967 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.121969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.121978 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:25.122003 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.122004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.122015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.122098 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.122384 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.122842 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.123526 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.123715 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.123819 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.126146 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:48573: BlockingRecv error: recv got EOF from 127.17.130.2:48573 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:25.130861 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.126437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:34955)
0812 01:09:25.126548 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:25.126559 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:25.126630 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:25.126750 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:25.126751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:25.126877 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:25.126904 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.126905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.127515 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.127516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.128006 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.128008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.128358 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.128359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.128411 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.128845 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:25.128850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.130771 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260812 01:09:25.130918 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:25.126489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:34955 (local address 127.17.130.62:34419)
0812 01:09:25.126548 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.126550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.126614 (+ 64us) server_negotiation.cc:407] Connection header received
0812 01:09:25.126690 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.126690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.126700 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.126748 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.126976 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.127433 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.128093 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.128282 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.128360 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.130853 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:34955: BlockingRecv error: recv got EOF from 127.17.130.2:34955 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 01:09:25.131006 32411 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:09:25.203924 32419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:25.204144 32419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:25.204178 32419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:25.205443 32419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:25.205498 32419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:25.205525 32419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:25.207038 32419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/krb5kdc/oryx_127.17.130.3.keytab
--principal=oryx/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:42361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=46361
--tserver_master_addrs=127.17.130.62:34419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:25.207319 32419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:25.207595 32419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:25.213215 32433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:25.213217 32419 server_base.cc:1034] running on GCE node
W20260812 01:09:25.213246 32429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:25.213246 32428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:25.213693 32419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:25.213966 32419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:25.215137 32419 hybrid_clock.cc:648] HybridClock initialized: now 1786496965215121 us; error 45 us; skew 500 ppm
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496965, etypes {rep=17 tkt=17 ses=17}, oryx/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:25.217844 32419 init.cc:377] Logged in from keytab as oryx/127.17.130.3@KRBTEST.COM (short username oryx)
I20260812 01:09:25.218360 32419 webserver.cc:460] Webserver started at http://127.17.130.3:46361/ using document root <none> and password file <none>
I20260812 01:09:25.218607 32419 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:25.218710 32419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:25.220248 32419 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:09:25.221048 32447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:25.221284 32419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:09:25.221410 32419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "892beaf7efaa470cacd538e3d8108cfb"
format_stamp: "Formatted at 2026-08-12 01:09:24 on dist-test-slave-tfpf"
server_key: "22add290bc6f411333c7ab184e56b2a3"
server_key_iv: "56b9e880f6df15bed427fc5c9b3c7573"
server_key_version: "encryptionkey@0"
I20260812 01:09:25.221802 32419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:25.244346 32419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:25.246959 32419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:25.247179 32419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:25.247498 32419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:25.247874 32419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:25.248073 32419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:25.248235 32419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:25.248286 32419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:25.255383 32419 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:42361
I20260812 01:09:25.255499 32647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:42361 every 8 connection(s)
I20260812 01:09:25.255904 32419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:25.261924 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 32419
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
W20260812 01:09:25.265161 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:25.256846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:34443 (local address 127.17.130.62:34419)
0812 01:09:25.256928 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.256935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.257110 (+ 175us) server_negotiation.cc:407] Connection header received
0812 01:09:25.257644 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.257645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.257658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.257733 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.258778 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.259283 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.260402 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.260607 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.260690 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.265091 (+ 4401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:34443: BlockingRecv error: recv got EOF from 127.17.130.3:34443 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:09:25.265192 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.256762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:34443)
0812 01:09:25.256931 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:25.256976 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:25.257513 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:25.257793 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:25.257796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:25.258413 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:25.258630 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.258636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.259419 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.259420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.260275 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.260278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.260744 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.260746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.260874 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.261910 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:25.261920 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.264945 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260812 01:09:25.265527 32648 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:34419 (0 consecutive failures): Not authorized: Failed to ping master at 127.17.130.62:34419: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:25.271067 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.265809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:42443)
0812 01:09:25.265852 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:25.265863 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:25.265908 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:25.266055 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:25.266056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:25.266203 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:25.266254 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.266256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.267004 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.267005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.267893 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.267897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.268408 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.268409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.268478 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.268891 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:25.268896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.270961 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 01:09:25.271122 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:25.265862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:42443 (local address 127.17.130.62:34419)
0812 01:09:25.265931 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.265933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.265942 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:25.265965 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.265966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.265976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.266057 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.266341 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.266899 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.267986 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.268316 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.268381 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.271052 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:42443: BlockingRecv error: recv got EOF from 127.17.130.3:42443 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 01:09:25.271822 32192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.262629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.17.130.62:34419)
0812 01:09:25.262725 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.262727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.262739 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:09:25.262807 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.262808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.262824 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.262906 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.263718 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.264401 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.265381 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.265697 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.265772 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.267842 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:25.267847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:25.267847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:25.267863 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:25.270079 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:25.270552 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:25.270556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:25.270557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:25.270719 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:25.270953 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:25.270954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:25.270955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:25.271292 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:25.271463 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:25.271577 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 01:09:25.271709 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:25.278148 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:25.272156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:52219)
0812 01:09:25.272404 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:25.272424 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:25.272512 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:25.272831 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:25.272832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:25.273294 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:25.273348 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.273350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.274043 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.274044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.274565 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:25.274567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.275403 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.275404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:25.275492 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.276022 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:25.276028 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:25.278042 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221}
W20260812 01:09:25.278208 32188 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:25.272222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:52219 (local address 127.17.130.62:34419)
0812 01:09:25.272366 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:25.272368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:25.272490 (+ 122us) server_negotiation.cc:407] Connection header received
0812 01:09:25.272642 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:25.272642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:25.272654 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:25.272822 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:25.273439 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.273951 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.274886 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:25.275246 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:25.275401 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:25.278140 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:52219: BlockingRecv error: recv got EOF from 127.17.130.3:52219 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260812 01:09:25.278394 32648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 01:09:25 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:26 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:26.006140 32654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:26.001099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:46289)
0812 01:09:26.001264 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:26.001274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:26.001326 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:26.001470 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:26.001470 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:26.001649 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:26.001704 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:26.001706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.002360 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.002361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:26.002930 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:26.002932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.003365 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.003366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:26.003417 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:26.003961 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:26.003967 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:26.006062 (+ 2095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
W20260812 01:09:26.006327 32656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:26.001199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:46289 (local address 127.17.130.62:34419)
0812 01:09:26.001337 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:26.001339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:26.001347 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:26.001378 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:26.001379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:26.001390 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:26.001452 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:26.001774 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.002279 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.003037 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.003252 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.003318 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:26.006231 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:46289: BlockingRecv error: recv got EOF from 127.17.130.1:46289 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
Aug 12 01:09:26 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:26 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:26.137837 32657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:26.131513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:43939)
0812 01:09:26.131730 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:26.131754 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:26.131860 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:26.132031 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:26.132032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:26.132224 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:26.132276 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:26.132278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.132996 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.132997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:26.133616 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:26.133618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.134006 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.134007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:26.134071 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:26.134590 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:26.134595 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:26.137764 (+ 3169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
W20260812 01:09:26.138000 32656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:26.131640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:43939 (local address 127.17.130.62:34419)
0812 01:09:26.131757 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:26.131760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:26.131845 (+ 85us) server_negotiation.cc:407] Connection header received
0812 01:09:26.131934 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:26.131935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:26.131949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:26.132012 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:26.132369 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.132888 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.133706 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.133907 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.134005 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:26.137900 (+ 3895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:43939: BlockingRecv error: recv got EOF from 127.17.130.2:43939 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 01:09:26 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:26 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:26.284866 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:26.279107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:54113)
0812 01:09:26.279308 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:26.279320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:26.279402 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:26.279680 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:26.279681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:26.279874 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:26.279929 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:26.279930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.280683 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.280685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:26.281502 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:26.281505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.281905 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.281906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:26.282014 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:26.282649 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:26.282654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:26.284793 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":104,"threads_started":1}
W20260812 01:09:26.285029 32656 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:26.279247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:54113 (local address 127.17.130.62:34419)
0812 01:09:26.279312 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:26.279314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:26.279454 (+ 140us) server_negotiation.cc:407] Connection header received
0812 01:09:26.279496 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:26.279497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:26.279512 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:26.279641 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:26.280082 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.280587 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.281602 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:26.281802 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:26.281869 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:26.284916 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:54113: BlockingRecv error: recv got EOF from 127.17.130.3:54113 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 01:09:27 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:27 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:27.012025 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:27.006725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:33219)
0812 01:09:27.006922 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:27.006943 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:27.007011 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:27.007228 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:27.007229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:27.007427 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:27.007481 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:27.007483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.008256 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.008257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:27.008835 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:27.008837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.009274 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.009275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:27.009323 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:27.009850 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:27.009855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:27.011917 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":120,"threads_started":1}
W20260812 01:09:27.012121 32661 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:27.006884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:33219 (local address 127.17.130.62:34419)
0812 01:09:27.007048 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:27.007050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:27.007060 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:27.007087 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:27.007088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:27.007100 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:27.007177 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:27.007632 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.008159 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.008962 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.009194 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.009261 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:27.012016 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:33219: BlockingRecv error: recv got EOF from 127.17.130.1:33219 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
Aug 12 01:09:27 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:27 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:27.145458 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:27.138436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:52145)
0812 01:09:27.138682 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:27.138698 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:27.138767 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:27.139242 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:27.139244 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:27.139437 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:27.139514 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:27.139516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.140510 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.140511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:27.141269 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:27.141272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.141794 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.141795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:27.141849 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:27.142384 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:27.142389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:27.145392 (+ 3003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
W20260812 01:09:27.145661 32661 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:27.138973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:52145 (local address 127.17.130.62:34419)
0812 01:09:27.139060 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:27.139062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:27.139071 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:27.139115 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:27.139116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:27.139128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:27.139177 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:27.139696 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.140404 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.141385 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.141687 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.141759 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:27.145541 (+ 3782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:52145: BlockingRecv error: recv got EOF from 127.17.130.2:52145 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 01:09:27 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:27 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:27.290683 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:27.285581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:50121)
0812 01:09:27.285759 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:27.285770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:27.285862 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:27.286065 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:27.286066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:27.286259 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:27.286310 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:27.286312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.287015 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.287016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:27.287551 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:27.287553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.287913 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.287915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:27.287979 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:27.288535 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:27.288540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:27.290569 (+ 2029us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
W20260812 01:09:27.290721 32661 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:27.285765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:50121 (local address 127.17.130.62:34419)
0812 01:09:27.285835 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:27.285837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:27.285848 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:27.285946 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:27.285948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:27.285966 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:27.286051 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:27.286403 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.286927 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.287655 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:27.287838 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:27.287914 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:27.290656 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:50121: BlockingRecv error: recv got EOF from 127.17.130.3:50121 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 12 01:09:28 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:28 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:28.018428 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:28.012649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:46907)
0812 01:09:28.012831 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:28.012842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:28.012908 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:28.013207 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:28.013208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:28.013383 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:28.013436 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:28.013438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.014282 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.014282 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:28.015007 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:28.015010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.015438 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.015439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:28.015491 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:28.016203 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:28.016209 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:28.018321 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":128,"threads_started":1}
W20260812 01:09:28.018491 32671 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:28.012790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:46907 (local address 127.17.130.62:34419)
0812 01:09:28.012940 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:28.012942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:28.012951 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:28.012979 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:28.012979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:28.012990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:28.013052 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:28.013548 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.014167 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.015125 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.015326 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.015399 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:28.018417 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:46907: BlockingRecv error: recv got EOF from 127.17.130.1:46907 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
Aug 12 01:09:28 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:28 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:28.151309 32672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:28.146034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:56173)
0812 01:09:28.146201 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:28.146220 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:28.146294 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:28.146562 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:28.146563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:28.146748 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:28.146800 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:28.146802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.147544 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.147545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:28.148086 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:28.148087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.148476 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.148477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:28.148524 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:28.149166 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:28.149172 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:28.151209 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
W20260812 01:09:28.151471 32671 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:28.146165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:56173 (local address 127.17.130.62:34419)
0812 01:09:28.146270 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:28.146272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:28.146282 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:28.146415 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:28.146417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:28.146434 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:28.146504 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:28.146906 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.147453 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.148201 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.148397 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.148462 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:28.151404 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:56173: BlockingRecv error: recv got EOF from 127.17.130.2:56173 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 01:09:28 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:28 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:28.296402 32673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:28.291342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:40549)
0812 01:09:28.291545 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:28.291558 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:28.291634 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:28.291822 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:28.291823 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:28.292053 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:28.292108 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:28.292111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.292827 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.292828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:28.293403 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:28.293405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.293783 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.293784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:28.293833 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:28.294353 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:28.294357 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:28.296337 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
W20260812 01:09:28.296514 32671 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:28.291449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:40549 (local address 127.17.130.62:34419)
0812 01:09:28.291529 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:28.291531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:28.291626 (+ 95us) server_negotiation.cc:407] Connection header received
0812 01:09:28.291692 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:28.291693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:28.291707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:28.291760 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:28.292213 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.292728 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.293503 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:28.293702 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:28.293769 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:28.296443 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:40549: BlockingRecv error: recv got EOF from 127.17.130.3:40549 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 01:09:29 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:29 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:29.027855 32674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:29.019091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:48743)
0812 01:09:29.019243 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:29.019255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:29.019318 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:29.019469 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:29.019470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:29.019649 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:29.019702 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:29.019704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.020424 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.020425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:29.021052 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:29.021054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.021543 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.021544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:29.021616 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:29.025645 (+ 4029us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:29.025651 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:29.027776 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260812 01:09:29.028080 32677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:29.019129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:48743 (local address 127.17.130.62:34419)
0812 01:09:29.019281 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:29.019283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:29.019305 (+ 22us) server_negotiation.cc:407] Connection header received
0812 01:09:29.019376 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:29.019377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:29.019388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:29.019441 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:29.019798 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.020323 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.021180 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.021381 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.021455 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:29.027906 (+ 6451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:48743: BlockingRecv error: recv got EOF from 127.17.130.1:48743 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
Aug 12 01:09:29 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:29 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:29.161046 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:29.151947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:57127)
0812 01:09:29.152145 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:29.152222 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:29.152291 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:29.152447 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:29.152448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:29.152760 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:29.152825 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:29.152828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.154024 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.154025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:29.154727 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:29.154730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.155403 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.155404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:29.155545 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:29.157379 (+ 1834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:29.157386 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:29.160961 (+ 3575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":69,"threads_started":1}
W20260812 01:09:29.161293 32677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:29.152129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:57127 (local address 127.17.130.62:34419)
0812 01:09:29.152172 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:29.152174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:29.152280 (+ 106us) server_negotiation.cc:407] Connection header received
0812 01:09:29.152360 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:29.152361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:29.152391 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:29.152440 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:29.152908 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.153890 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.154815 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.155307 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.155403 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:29.161202 (+ 5799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:57127: BlockingRecv error: recv got EOF from 127.17.130.2:57127 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 01:09:29 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:29 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:29.302613 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:29.297204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:34661)
0812 01:09:29.297359 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:29.297369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:29.297463 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:29.297627 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:29.297628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:29.297829 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:29.297882 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:29.297884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.298636 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.298638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:29.299170 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:29.299172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.299592 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.299593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:29.299664 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:29.300224 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:29.300229 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:29.302546 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
W20260812 01:09:29.302807 32677 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:29.297211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:34661 (local address 127.17.130.62:34419)
0812 01:09:29.297298 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:29.297301 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:29.297447 (+ 146us) server_negotiation.cc:407] Connection header received
0812 01:09:29.297533 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:29.297534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:29.297555 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:29.297619 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:29.297974 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.298541 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.299266 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:29.299503 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:29.299590 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:29.302728 (+ 3138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:34661: BlockingRecv error: recv got EOF from 127.17.130.3:34661 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 01:09:30 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:30 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:30.033986 32683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:30.028587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:39351)
0812 01:09:30.028790 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:30.028800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:30.028855 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:30.029124 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:30.029126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:30.029333 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:30.029411 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:30.029413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.030198 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.030199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:30.030764 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:30.030766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.031180 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.031181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:30.031233 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:30.031813 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:30.031819 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:30.033865 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":127,"threads_started":1}
W20260812 01:09:30.034108 32685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:30.028719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:39351 (local address 127.17.130.62:34419)
0812 01:09:30.028890 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:30.028893 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:30.028902 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:30.028935 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:30.028936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:30.028951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:30.029037 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:30.029551 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.030094 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.030868 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.031075 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.031146 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:30.034013 (+ 2867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:39351: BlockingRecv error: recv got EOF from 127.17.130.1:39351 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
Aug 12 01:09:30 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:30 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:30.167529 32686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:30.161693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:36905)
0812 01:09:30.161932 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:30.161946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:30.162043 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:30.162350 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:30.162351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:30.162535 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:30.162587 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:30.162588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.163822 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.163823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:30.164470 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:30.164474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.164876 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.164877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:30.164938 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:30.165411 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:30.165420 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:30.167388 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":100,"threads_started":1}
W20260812 01:09:30.167586 32685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:30.161897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:36905 (local address 127.17.130.62:34419)
0812 01:09:30.162104 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:30.162106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:30.162116 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:30.162142 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:30.162143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:30.162205 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:30.162283 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:30.162699 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.163709 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.164577 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.164773 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.164847 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:30.167538 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:36905: BlockingRecv error: recv got EOF from 127.17.130.2:36905 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 12 01:09:30 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:30 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:30.310470 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:30.303485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:47415)
0812 01:09:30.303675 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:30.303692 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:30.303877 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:30.304171 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:30.304172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:30.304631 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:30.304846 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:30.304851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.305731 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.305740 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:30.306308 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:30.306310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.306759 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.306760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:30.306963 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:30.307709 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:30.307715 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:30.310384 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
W20260812 01:09:30.310674 32685 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:30.303566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:47415 (local address 127.17.130.62:34419)
0812 01:09:30.303640 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:30.303642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:30.303894 (+ 252us) server_negotiation.cc:407] Connection header received
0812 01:09:30.303942 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:30.303943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:30.303959 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:30.304026 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:30.305029 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.305594 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.306446 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:30.306645 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:30.306720 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:30.310598 (+ 3878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:47415: BlockingRecv error: recv got EOF from 127.17.130.3:47415 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 01:09:31 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:31 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:31.039922 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:31.034567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:41813)
0812 01:09:31.034773 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:31.034786 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:31.034860 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:31.035113 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:31.035114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:31.035302 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:31.035355 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:31.035357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.036202 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.036203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:31.036821 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:31.036824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.037280 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.037281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:31.037339 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:31.037920 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:31.037926 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:31.039853 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":104,"threads_started":1}
W20260812 01:09:31.040333 32691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:31.034721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:41813 (local address 127.17.130.62:34419)
0812 01:09:31.034875 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:31.034877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:31.034888 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:31.034914 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:31.034915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:31.034926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:31.035001 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:31.035457 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.036098 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.036938 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.037189 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.037250 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:31.040266 (+ 3016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:41813: BlockingRecv error: recv got EOF from 127.17.130.1:41813 (error 108)
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
Aug 12 01:09:31 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:31 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:31.173532 32693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:31.168219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:38099)
0812 01:09:31.168450 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:31.168463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:31.168578 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:31.168763 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:31.168763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:31.168974 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:31.169023 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:31.169025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.169749 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.169750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:31.170285 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:31.170286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.170667 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.170668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:31.170725 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:31.171249 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:31.171255 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:31.173431 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":137,"threads_started":1}
W20260812 01:09:31.173638 32691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:31.168360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:38099 (local address 127.17.130.62:34419)
0812 01:09:31.168450 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:31.168453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:31.168545 (+ 92us) server_negotiation.cc:407] Connection header received
0812 01:09:31.168666 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:31.168667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:31.168680 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:31.168754 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:31.169154 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.169660 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.170373 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.170576 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.170666 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:31.173567 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:38099: BlockingRecv error: recv got EOF from 127.17.130.2:38099 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 01:09:31 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:31 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:31.318222 32695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:31.311230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:39235)
0812 01:09:31.311487 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:31.311501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:31.311617 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:31.311809 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:31.311810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:31.312039 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:31.312151 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:31.312154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.313217 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.313218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:31.313942 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:31.313946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.314466 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.314468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:31.314521 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:31.315552 (+ 1031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:31.315557 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:31.318153 (+ 2596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":142,"threads_started":1}
W20260812 01:09:31.318454 32691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:31.311364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:39235 (local address 127.17.130.62:34419)
0812 01:09:31.311471 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:31.311474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:31.311616 (+ 142us) server_negotiation.cc:407] Connection header received
0812 01:09:31.311696 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:31.311697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:31.311713 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:31.311781 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:31.312296 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.313140 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.314101 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:31.314348 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:31.314435 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:31.318300 (+ 3865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:39235: BlockingRecv error: recv got EOF from 127.17.130.3:39235 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 01:09:32 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:32 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:32.046862 32697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:32.040596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:60959)
0812 01:09:32.040791 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:32.040805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:32.040874 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:32.041244 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:32.041245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:32.041430 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:32.041485 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:32.041487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.042285 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.042286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:32.042834 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:32.042836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.043310 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.043311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:32.043358 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:32.044069 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:32.044074 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:32.046788 (+ 2714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":126,"threads_started":1}
W20260812 01:09:32.047289 32700 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:32.040731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:60959 (local address 127.17.130.62:34419)
0812 01:09:32.041097 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:32.041099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:32.041108 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:32.041137 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:32.041137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:32.041153 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:32.041223 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:32.041609 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.042166 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.042998 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.043227 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.043292 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:32.047206 (+ 3914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:60959: BlockingRecv error: recv got EOF from 127.17.130.1:60959 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":69,"threads_started":1}
Aug 12 01:09:32 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:32 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:32.180558 32701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:32.174253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:58387)
0812 01:09:32.174417 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:32.174435 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:32.174519 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:32.174707 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:32.174708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:32.174902 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:32.174964 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:32.174966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.175749 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.175750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:32.176361 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:32.176364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.177009 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.177010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:32.177119 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:32.177714 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:32.177720 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:32.180449 (+ 2729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
W20260812 01:09:32.180778 32700 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:32.174350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58387 (local address 127.17.130.62:34419)
0812 01:09:32.174463 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:32.174465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:32.174518 (+ 53us) server_negotiation.cc:407] Connection header received
0812 01:09:32.174573 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:32.174574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:32.174587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:32.174664 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:32.175078 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.175634 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.176541 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.176855 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.176974 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:32.180666 (+ 3692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:58387: BlockingRecv error: recv got EOF from 127.17.130.2:58387 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 12 01:09:32 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:32 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:32.325111 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:32.319016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:37163)
0812 01:09:32.319187 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:32.319203 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:32.319262 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:32.319582 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:32.319584 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:32.319827 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:32.319900 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:32.319903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.321016 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.321022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:32.321663 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:32.321666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.322113 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.322114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:32.322166 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:32.322659 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:32.322664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:32.324985 (+ 2321us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
W20260812 01:09:32.325191 32700 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:32.319167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:37163 (local address 127.17.130.62:34419)
0812 01:09:32.319301 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:32.319304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:32.319326 (+ 22us) server_negotiation.cc:407] Connection header received
0812 01:09:32.319373 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:32.319374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:32.319391 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:32.319491 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:32.320017 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.320770 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.321775 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:32.321981 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:32.322076 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:32.325135 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:37163: BlockingRecv error: recv got EOF from 127.17.130.3:37163 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 01:09:33 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:33 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:33.053217 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:33.047454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:54941)
0812 01:09:33.047656 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:33.047668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:33.047725 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:33.048066 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:33.048067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:33.048259 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:33.048312 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:33.048314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.049102 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.049103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:33.049638 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:33.049640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.050177 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.050178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:33.050232 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:33.050732 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:33.050738 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:33.053139 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
W20260812 01:09:33.053433 32707 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:33.047685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:54941 (local address 127.17.130.62:34419)
0812 01:09:33.047871 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:33.047873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:33.047882 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:33.047912 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:33.047913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:33.047924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:33.048002 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:33.048427 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.048944 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.049808 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.050068 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.050147 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:33.053338 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:54941: BlockingRecv error: recv got EOF from 127.17.130.1:54941 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":109,"threads_started":1}
Aug 12 01:09:33 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:33 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:33.187229 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:33.181276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:57781)
0812 01:09:33.181415 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:33.181426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:33.181479 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:33.181861 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:33.181862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:33.182070 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:33.182145 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:33.182147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.183028 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.183029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:33.183621 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:33.183622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.184025 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.184027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:33.184078 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:33.184656 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:33.184661 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:33.187160 (+ 2499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
W20260812 01:09:33.187450 32707 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:33.181444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:57781 (local address 127.17.130.62:34419)
0812 01:09:33.181570 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:33.181572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:33.181599 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:09:33.181642 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:33.181643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:33.181663 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:33.181837 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:33.182253 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.182906 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.183732 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.183936 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.184005 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:33.187352 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:57781: BlockingRecv error: recv got EOF from 127.17.130.2:57781 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 12 01:09:33 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:33 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:33.331092 32709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:33.325802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:33569)
0812 01:09:33.325968 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:33.325986 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:33.326053 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:33.326260 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:33.326261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:33.326440 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:33.326490 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:33.326492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.327239 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.327240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:33.327777 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:33.327779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.328193 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.328194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:33.328245 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:33.328784 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:33.328788 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:33.331016 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
W20260812 01:09:33.331277 32707 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:33.325895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:33569 (local address 127.17.130.62:34419)
0812 01:09:33.325978 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:33.325980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:33.326038 (+ 58us) server_negotiation.cc:407] Connection header received
0812 01:09:33.326104 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:33.326105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:33.326118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:33.326176 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:33.326607 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.327140 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.327884 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:33.328082 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:33.328147 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:33.331198 (+ 3051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:33569: BlockingRecv error: recv got EOF from 127.17.130.3:33569 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 01:09:34 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:34 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:34.064033 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:34.053843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:55479)
0812 01:09:34.054048 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:34.054060 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:34.054127 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:34.054284 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:34.054285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:34.054464 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:34.054517 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:34.054519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.055363 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.055364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:34.056006 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:34.056009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.056442 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.056444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:34.056495 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:34.061786 (+ 5291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:34.061793 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:34.063962 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
W20260812 01:09:34.064257 32716 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:34.053995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:55479 (local address 127.17.130.62:34419)
0812 01:09:34.054133 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:34.054135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:34.054148 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:34.054175 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:34.054176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:34.054190 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:34.054268 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:34.054609 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.055175 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.056124 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.056323 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.056393 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:34.064187 (+ 7794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:55479: BlockingRecv error: recv got EOF from 127.17.130.1:55479 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
Aug 12 01:09:34 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:34 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:34.192970 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:34.187773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:49113)
0812 01:09:34.187965 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:34.187985 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:34.188061 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:34.188207 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:34.188208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:34.188390 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:34.188444 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:34.188445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.189308 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.189309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:34.189933 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:34.189935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.190408 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.190409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:34.190487 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:34.190982 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:34.190987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:34.192890 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
W20260812 01:09:34.193198 32716 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:34.187915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:49113 (local address 127.17.130.62:34419)
0812 01:09:34.187974 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:34.187976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:34.188039 (+ 63us) server_negotiation.cc:407] Connection header received
0812 01:09:34.188106 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:34.188106 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:34.188117 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:34.188182 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:34.188544 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.189213 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.190017 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.190218 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.190402 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:34.193100 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:49113: BlockingRecv error: recv got EOF from 127.17.130.2:49113 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 01:09:34 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:34 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:34.337543 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:34.331736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:34219)
0812 01:09:34.331893 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:34.331904 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:34.332136 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:34.332313 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:34.332314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:34.332549 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:34.332612 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:34.332615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.333352 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.333353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:34.333912 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:34.333914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.334314 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.334315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:34.334371 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:34.334889 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:34.334894 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:34.337468 (+ 2574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
W20260812 01:09:34.337728 32716 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:34.331897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:34219 (local address 127.17.130.62:34419)
0812 01:09:34.331980 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:34.331982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:34.332111 (+ 129us) server_negotiation.cc:407] Connection header received
0812 01:09:34.332207 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:34.332208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:34.332224 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:34.332300 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:34.332682 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.333269 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.334023 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:34.334233 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:34.334315 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:34.337662 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:34219: BlockingRecv error: recv got EOF from 127.17.130.3:34219 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 01:09:35 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:35 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:35.070307 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:35.064680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:42485)
0812 01:09:35.064838 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:35.064855 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:35.064910 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:35.065251 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:35.065252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:35.065443 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:35.065497 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:35.065499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.066206 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.066207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:35.066752 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:35.066754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.067145 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.067146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:35.067203 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:35.067734 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:35.067739 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:35.070200 (+ 2461us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
W20260812 01:09:35.070355 32720 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:35.064767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:42485 (local address 127.17.130.62:34419)
0812 01:09:35.064945 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:35.064947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:35.064956 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:35.064982 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:35.064983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:35.064995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:35.065245 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:35.065585 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.066092 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.066842 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.067040 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.067106 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:35.070285 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:42485: BlockingRecv error: recv got EOF from 127.17.130.1:42485 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
Aug 12 01:09:35 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:35 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:35.200582 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:35.193596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:39225)
0812 01:09:35.193778 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:35.193790 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:35.193847 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:35.194031 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:35.194032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:35.194240 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:35.194292 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:35.194294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.195135 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.195136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:35.195884 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:35.195889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.196532 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.196533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:35.196584 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:35.197235 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:35.197241 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:35.200483 (+ 3242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
W20260812 01:09:35.200791 32720 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:35.193756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:39225 (local address 127.17.130.62:34419)
0812 01:09:35.193832 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:35.193834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:35.193846 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:09:35.193897 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:35.193898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:35.193911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:35.193969 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:35.194425 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.195014 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.196079 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.196378 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.196466 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:35.200716 (+ 4250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:39225: BlockingRecv error: recv got EOF from 127.17.130.2:39225 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 01:09:35 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:35 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:35.344216 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:35.338360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:58463)
0812 01:09:35.338504 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:35.338519 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:35.338596 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:35.338824 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:35.338825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:35.339073 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:35.339135 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:35.339137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.340002 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.340004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:35.340639 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:35.340642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.341251 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.341252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:35.341318 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:35.341903 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:35.341908 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:35.344125 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
W20260812 01:09:35.344408 32720 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:35.338481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:58463 (local address 127.17.130.62:34419)
0812 01:09:35.338562 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:35.338565 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:35.338577 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:09:35.338680 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:35.338681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:35.338697 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:35.338779 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:35.339222 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.339826 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.340844 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:35.341111 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:35.341231 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:35.344303 (+ 3072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:58463: BlockingRecv error: recv got EOF from 127.17.130.3:58463 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 01:09:36 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:36 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:36.076453 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:36.070965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:35969)
0812 01:09:36.071164 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:36.071179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:36.071264 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:36.071442 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:36.071443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:36.071630 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:36.071692 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:36.071693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.072607 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.072608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:36.073312 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:36.073316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.073726 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.073727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:36.073779 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:36.074377 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:36.074383 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:36.076334 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
W20260812 01:09:36.076510 32724 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:36.071096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:35969 (local address 127.17.130.62:34419)
0812 01:09:36.071302 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:36.071303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:36.071313 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:36.071341 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:36.071341 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:36.071353 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:36.071419 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:36.071762 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.072487 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.073413 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.073622 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.073695 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:36.076424 (+ 2729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:35969: BlockingRecv error: recv got EOF from 127.17.130.1:35969 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":137,"threads_started":1}
Aug 12 01:09:36 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:36 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:36.206780 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:36.201305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:34469)
0812 01:09:36.201481 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:36.201494 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:36.201602 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:36.201766 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:36.201767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:36.202007 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:36.202060 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:36.202063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.202786 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.202788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:36.203352 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:36.203354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.203764 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.203766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:36.203830 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:36.204461 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:36.204467 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:36.206704 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
W20260812 01:09:36.206944 32724 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:36.201401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:34469 (local address 127.17.130.62:34419)
0812 01:09:36.201481 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:36.201484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:36.201578 (+ 94us) server_negotiation.cc:407] Connection header received
0812 01:09:36.201680 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:36.201681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:36.201699 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:36.201758 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:36.202163 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.202689 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.203457 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.203666 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.203763 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:36.206837 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:34469: BlockingRecv error: recv got EOF from 127.17.130.2:34469 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 12 01:09:36 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:36 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:36.350378 32726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:36.344816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:40633)
0812 01:09:36.345144 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:36.345158 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:36.345247 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:36.345454 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:36.345456 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:36.345650 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:36.345710 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:36.345712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.346441 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.346443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:36.347075 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:36.347076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.347490 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.347491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:36.347541 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:36.348100 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:36.348105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:36.350252 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":204,"threads_started":1}
W20260812 01:09:36.350419 32724 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:36.345205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:40633 (local address 127.17.130.62:34419)
0812 01:09:36.345278 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:36.345281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:36.345291 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:36.345329 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:36.345330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:36.345345 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:36.345407 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:36.345820 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.346337 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.347190 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:36.347375 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:36.347447 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:36.350347 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:40633: BlockingRecv error: recv got EOF from 127.17.130.3:40633 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 01:09:37 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:37 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:37.083262 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:37.077228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:50581)
0812 01:09:37.077401 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:37.077416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:37.077521 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:37.077899 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:37.077900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:37.078182 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:37.078233 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:37.078235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.079330 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.079331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:37.080118 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:37.080122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.080563 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.080564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:37.080630 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:37.081193 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:37.081199 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:37.083200 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
W20260812 01:09:37.083461 32728 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:37.077370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:50581 (local address 127.17.130.62:34419)
0812 01:09:37.077557 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:37.077559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:37.077572 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:37.077600 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:37.077601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:37.077614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:37.077889 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:37.078328 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.079202 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.080257 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.080460 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.080560 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:37.083374 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:50581: BlockingRecv error: recv got EOF from 127.17.130.1:50581 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
Aug 12 01:09:37 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:37 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:37.213181 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:37.207496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:48811)
0812 01:09:37.207700 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:37.207714 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:37.207789 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:37.208070 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:37.208071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:37.208278 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:37.208331 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:37.208333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.209169 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.209170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:37.209922 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:37.209926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.210622 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.210623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:37.210673 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:37.211261 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:37.211267 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:37.213048 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":86,"threads_started":1}
W20260812 01:09:37.213303 32728 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:37.207631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:48811 (local address 127.17.130.62:34419)
0812 01:09:37.207766 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:37.207768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:37.207780 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:09:37.207894 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:37.207895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:37.207920 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:37.208002 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:37.208459 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.208966 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.210199 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.210523 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.210603 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:37.213216 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:48811: BlockingRecv error: recv got EOF from 127.17.130.2:48811 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 12 01:09:37 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:37 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:37.356145 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:37.350900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:59205)
0812 01:09:37.351027 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:37.351039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:37.351105 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:37.351238 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:37.351239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:37.351471 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:37.351518 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:37.351520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.352270 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.352271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:37.352887 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:37.352890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.353363 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.353363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:37.353422 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:37.354005 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:37.354011 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:37.356026 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
W20260812 01:09:37.356218 32728 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:37.351036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:59205 (local address 127.17.130.62:34419)
0812 01:09:37.351125 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:37.351127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:37.351136 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:37.351166 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:37.351167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:37.351181 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:37.351233 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:37.351605 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.352184 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.352980 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:37.353280 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:37.353362 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:37.356152 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:59205: BlockingRecv error: recv got EOF from 127.17.130.3:59205 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 01:09:38 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:38 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:38.089192 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:38.083869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:36823)
0812 01:09:38.084080 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:38.084090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:38.084142 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:38.084327 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:38.084328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:38.084527 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:38.084590 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:38.084592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.085266 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.085267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:38.085848 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:38.085850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.086273 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.086274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:38.086327 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:38.086849 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:38.086854 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:38.089044 (+ 2190us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":116,"threads_started":1}
W20260812 01:09:38.089552 32736 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:38.084031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:36823 (local address 127.17.130.62:34419)
0812 01:09:38.084181 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:38.084183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:38.084198 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:09:38.084223 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:38.084223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:38.084235 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:38.084307 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:38.084671 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.085199 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.085952 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.086178 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.086247 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:38.089386 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:36823: BlockingRecv error: recv got EOF from 127.17.130.1:36823 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
Aug 12 01:09:38 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:38 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:38.219290 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:38.213892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:36857)
0812 01:09:38.214072 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:38.214091 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:38.214143 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:38.214287 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:38.214288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:38.214470 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:38.214521 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:38.214523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.215249 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.215251 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:38.215801 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:38.215803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.216256 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.216257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:38.216308 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:38.216886 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:38.216891 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:38.219223 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":125,"threads_started":1}
W20260812 01:09:38.219444 32736 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:38.214075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:36857 (local address 127.17.130.62:34419)
0812 01:09:38.214150 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:38.214152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:38.214163 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:38.214201 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:38.214202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:38.214213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:38.214271 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:38.214632 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.215137 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.215921 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.216162 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.216235 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:38.219355 (+ 3120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:36857: BlockingRecv error: recv got EOF from 127.17.130.2:36857 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 01:09:38 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:38 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:38.362181 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:38.356771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:45031)
0812 01:09:38.357184 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:38.357198 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:38.357270 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:38.357425 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:38.357426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:38.357600 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:38.357656 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:38.357657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.358299 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.358300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:38.358832 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:38.358833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.359495 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.359496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:38.359556 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:38.360249 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:38.360256 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:38.362109 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":99,"threads_started":1}
W20260812 01:09:38.362358 32736 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:38.357135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:45031 (local address 127.17.130.62:34419)
0812 01:09:38.357294 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:38.357296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:38.357306 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:38.357334 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:38.357335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:38.357347 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:38.357405 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:38.357725 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.358222 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.358945 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:38.359365 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:38.359442 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:38.362247 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:45031: BlockingRecv error: recv got EOF from 127.17.130.3:45031 (error 108)
Metrics: {"server-negotiator.queue_time_us":122}
Aug 12 01:09:39 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:39 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:39.098711 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:39.089847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:43893)
0812 01:09:39.090075 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:39.090088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:39.090143 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:39.090328 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:39.090329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:39.090526 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:39.090577 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:39.090580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.091290 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.091291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:39.091826 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:39.091828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.092239 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.092240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:39.092291 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:39.096272 (+ 3981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:39.096279 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:39.098594 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":127,"threads_started":1}
W20260812 01:09:39.098767 32744 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:39.089984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:43893 (local address 127.17.130.62:34419)
0812 01:09:39.090117 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:39.090119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:39.090147 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:09:39.090203 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:39.090203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:39.090214 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:39.090274 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:39.090666 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.091213 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.091931 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.092147 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.092213 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:39.098694 (+ 6481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:43893: BlockingRecv error: recv got EOF from 127.17.130.1:43893 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
Aug 12 01:09:39 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:39 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:39.226584 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:39.219880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:37947)
0812 01:09:39.220124 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:39.220149 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:39.220227 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:39.220436 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:39.220437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:39.220641 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:39.220692 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:39.220694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.221872 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.221874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:39.222759 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:39.222763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.223330 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.223332 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:39.223410 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:39.224117 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:39.224123 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:39.226520 (+ 2397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":80,"threads_started":1}
W20260812 01:09:39.226821 32744 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:39.219936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:37947 (local address 127.17.130.62:34419)
0812 01:09:39.220001 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:39.220004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:39.220294 (+ 290us) server_negotiation.cc:407] Connection header received
0812 01:09:39.220327 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:39.220328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:39.220344 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:39.220405 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:39.220782 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.221755 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.222860 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.223218 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.223309 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:39.226734 (+ 3425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:37947: BlockingRecv error: recv got EOF from 127.17.130.2:37947 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 12 01:09:39 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:39 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.3@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:39.368391 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:39.362850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.3:44017)
0812 01:09:39.363050 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:39.363062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:39.363123 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:39.363335 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:39.363337 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:39.363535 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:39.363587 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:39.363589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.364535 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.364536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:39.365140 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:39.365141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.365528 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.365530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:39.365580 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:39.366361 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:39.366367 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:39.368312 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":108,"threads_started":1}
W20260812 01:09:39.368574 32744 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:39.362964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:44017 (local address 127.17.130.62:34419)
0812 01:09:39.363153 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:39.363155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:39.363168 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:09:39.363199 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:39.363200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:39.363214 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:39.363283 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:39.363693 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.364426 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.365235 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:39.365424 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:39.365498 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:39.368463 (+ 2965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.3:44017: BlockingRecv error: recv got EOF from 127.17.130.3:44017 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
Aug 12 01:09:40 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:40 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.1@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:40.104733 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.099430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.1:58857)
0812 01:09:40.099607 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:40.099623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:40.099722 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:40.099928 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:40.099929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:40.100114 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:40.100166 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.100168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.100919 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.100920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.101652 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.101655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.102206 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.102207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.102258 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.102832 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:40.102838 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:40.104659 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
W20260812 01:09:40.104947 32751 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:40.099484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:58857 (local address 127.17.130.62:34419)
0812 01:09:40.099656 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:40.099659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:40.099697 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:09:40.099804 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:40.099805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:40.099820 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:40.099902 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:40.100249 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.100814 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.101748 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.102104 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.102200 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.104860 (+ 2660us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.1:58857: BlockingRecv error: recv got EOF from 127.17.130.1:58857 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
Aug 12 01:09:40 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
Aug 12 01:09:40 dist-test-slave-tfpf krb5kdc[30967](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.17.130.2@KRBTEST.COM for oryx/127.17.130.62@KRBTEST.COM, Server not found in Kerberos database
I20260812 01:09:40.232496 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.227231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:34419 (local address 127.17.130.2:50579)
0812 01:09:40.227424 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:40.227437 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:40.227519 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:40.227695 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:40.227697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:40.227897 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:40.227949 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.227951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.228678 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.228679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.229420 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.229423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.229821 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.229822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.229863 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.230403 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:40.230408 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:40.232422 (+ 2014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.17.130.62:34419: Server oryx/127.17.130.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
W20260812 01:09:40.232683 32751 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:40.227354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:50579 (local address 127.17.130.62:34419)
0812 01:09:40.227438 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:40.227440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:40.227497 (+ 57us) server_negotiation.cc:407] Connection header received
0812 01:09:40.227588 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:40.227589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:40.227601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:40.227647 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:40.228055 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.228559 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.229525 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.229743 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.229814 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.232591 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.17.130.2:50579: BlockingRecv error: recv got EOF from 127.17.130.2:50579 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 01:09:40.266693 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 31924
I20260812 01:09:40.281548 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 32191
I20260812 01:09:40.298028 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 32419
I20260812 01:09:40.314900 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 31813
2026-08-12T01:09:40Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18090 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T01:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:40Z Disabled control of system clock
I20260812 01:09:40.362093 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:38113
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:38113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 01:09:40.458444 32758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:40.458691 32758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:40.458822 32758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:40.460491 32758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:40.460557 32758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:40.460572 32758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:40.460584 32758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:40.460597 32758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:40.462574 32758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:38113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:38113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:40.462888 32758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:40.463274 32758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:40.469386 32765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.469390 300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.469515 32766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:40.469555 32758 server_base.cc:1034] running on GCE node
I20260812 01:09:40.470026 32758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:40.470369 32758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:40.471578 32758 hybrid_clock.cc:648] HybridClock initialized: now 1786496980471556 us; error 41 us; skew 500 ppm
I20260812 01:09:40.473227 32758 webserver.cc:460] Webserver started at http://127.17.130.62:39985/ using document root <none> and password file <none>
I20260812 01:09:40.473500 32758 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:40.473580 32758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:40.473757 32758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:40.474969 32758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "e2d9d1b7ecee42a48219cc5bf2804f80"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "68f57800c22ab9cb990a1941f9fe3fa6"
server_key_iv: "339a058e95494054c901c1d215120a91"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.475391 32758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "e2d9d1b7ecee42a48219cc5bf2804f80"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "68f57800c22ab9cb990a1941f9fe3fa6"
server_key_iv: "339a058e95494054c901c1d215120a91"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.477279 32758 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:09:40.478642 318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.479030 32758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:40.479212 32758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "e2d9d1b7ecee42a48219cc5bf2804f80"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "68f57800c22ab9cb990a1941f9fe3fa6"
server_key_iv: "339a058e95494054c901c1d215120a91"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.479321 32758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:40.490046 32758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:40.490444 32758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:40.490617 32758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:40.495733 32758 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:38113
I20260812 01:09:40.495772 396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:38113 every 8 connection(s)
I20260812 01:09:40.496167 32758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:40.497417 398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:40.499229 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 32758
I20260812 01:09:40.499341 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:40.499728 17928 external_mini_cluster.cc:1428] Setting key 42df522ae80093e1b320336bd3d4158c
I20260812 01:09:40.502799 398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80: Bootstrap starting.
I20260812 01:09:40.503616 398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:40.503957 398 log.cc:824] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:40.504792 398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80: No bootstrap required, opened a new log
I20260812 01:09:40.506767 398 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 38113 } }
I20260812 01:09:40.506974 398 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:40.507020 398 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2d9d1b7ecee42a48219cc5bf2804f80, State: Initialized, Role: FOLLOWER
I20260812 01:09:40.507205 398 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 38113 } }
I20260812 01:09:40.507302 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.500821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.17.130.62:38113)
0812 01:09:40.501137 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:40.501147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:40.501186 (+ 39us) server_negotiation.cc:407] Connection header received
0812 01:09:40.501826 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:40.501836 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:40.501925 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:40.502088 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:40.502362 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.503732 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.504610 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.505241 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.505382 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.506162 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:40.506167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:40.506167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:40.506384 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:40.506430 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:40.506448 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:40.506525 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:40.506680 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 01:09:40.506767 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":127,"threads_started":1}
I20260812 01:09:40.507319 398 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:40.507357 398 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:40.507401 398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:40.508122 398 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 38113 } }
I20260812 01:09:40.508251 398 leader_election.cc:302] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2d9d1b7ecee42a48219cc5bf2804f80; no voters:
I20260812 01:09:40.508456 398 leader_election.cc:288] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:40.508535 405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:40.508688 405 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [term 1 LEADER]: Becoming Leader. State: Replica: e2d9d1b7ecee42a48219cc5bf2804f80, State: Running, Role: LEADER
I20260812 01:09:40.508771 398 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:40.508802 405 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 38113 } }
I20260812 01:09:40.509177 405 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2d9d1b7ecee42a48219cc5bf2804f80. Latest consensus state: current_term: 1 leader_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 38113 } } }
I20260812 01:09:40.509191 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d9d1b7ecee42a48219cc5bf2804f80" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 38113 } } }
I20260812 01:09:40.509379 405 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:40.509392 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80 [sys.catalog]: This master's current role is: LEADER
W20260812 01:09:40.510741 436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:09:40.510814 436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:09:40.510865 424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:40.511018 424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:40.512658 424 catalog_manager.cc:1347] Generated new cluster ID: 6a42dc547c8742698981a67b95f5757c
I20260812 01:09:40.512723 424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:40.525833 424 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:40.526592 424 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:40.534454 424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2d9d1b7ecee42a48219cc5bf2804f80: Generated new TSK 0
I20260812 01:09:40.534696 424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:09:40.543398 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:38113
--builtin_ntp_servers=127.17.130.20:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 01:09:40.651425 440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:40.651708 440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:40.651751 440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:40.653144 440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:40.653221 440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:40.653268 440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:40.654834 440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:40.655174 440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:40.655543 440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:40.663513 447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.663712 453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.663712 450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:40.664469 440 server_base.cc:1034] running on GCE node
I20260812 01:09:40.664811 440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:40.665215 440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:40.666432 440 hybrid_clock.cc:648] HybridClock initialized: now 1786496980666405 us; error 46 us; skew 500 ppm
I20260812 01:09:40.668208 440 webserver.cc:460] Webserver started at http://127.17.130.1:42807/ using document root <none> and password file <none>
I20260812 01:09:40.668540 440 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:40.668629 440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:40.668819 440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:40.670228 440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "4a69a9631b7340ba85a331aec6f39ec2"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "99204723a54c171624397e286c7a98a3"
server_key_iv: "88577f6da2c65f6b6de0fbc5ab007bfa"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.670749 440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "4a69a9631b7340ba85a331aec6f39ec2"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "99204723a54c171624397e286c7a98a3"
server_key_iv: "88577f6da2c65f6b6de0fbc5ab007bfa"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.672465 440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:09:40.673487 460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.673955 440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:40.674144 440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "4a69a9631b7340ba85a331aec6f39ec2"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "99204723a54c171624397e286c7a98a3"
server_key_iv: "88577f6da2c65f6b6de0fbc5ab007bfa"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.674245 440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:40.690536 440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:40.691051 440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:40.691238 440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:40.691524 440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:40.691931 440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:40.692003 440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.692042 440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:40.692070 440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.698709 440 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:44315
I20260812 01:09:40.698879 639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:44315 every 8 connection(s)
I20260812 01:09:40.699373 440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:40.701367 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 440
I20260812 01:09:40.701478 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:40.701712 17928 external_mini_cluster.cc:1428] Setting key b30a6d098f663d3c0e1354024650b289
I20260812 01:09:40.702771 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:38113
--builtin_ntp_servers=127.17.130.20:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 01:09:40.707618 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.700629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:33605 (local address 127.17.130.62:38113)
0812 01:09:40.700715 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:40.700717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:40.701351 (+ 634us) server_negotiation.cc:407] Connection header received
0812 01:09:40.702190 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:40.702192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:40.702205 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:40.702264 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:40.703111 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.703634 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.705433 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.705691 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.705795 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.707213 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:40.707214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:40.707214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:40.707220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:40.707235 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:40.707239 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:40.707298 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:40.707499 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 01:09:40.707547 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 01:09:40.707918 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.700638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:38113 (local address 127.17.130.1:33605)
0812 01:09:40.701175 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:40.701228 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:40.702047 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:40.702362 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:40.702366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:40.702376 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:40.703010 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.703017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.703745 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.703747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.705225 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.705236 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.705848 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.705850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.706093 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.707023 (+ 930us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:40.707023 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:40.707064 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:40.707097 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:40.707338 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:40.707346 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:40.707350 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:40.707437 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 01:09:40.707603 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":188,"threads_started":1}
I20260812 01:09:40.708616 640 heartbeater.cc:347] Connected to a master server at 127.17.130.62:38113
I20260812 01:09:40.708858 640 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:40.709178 640 heartbeater.cc:511] Master 127.17.130.62:38113 requested a full tablet report, sending...
I20260812 01:09:40.709856 344 ts_manager.cc:194] Registered new tserver with Master: 4a69a9631b7340ba85a331aec6f39ec2 (127.17.130.1:44315)
I20260812 01:09:40.710561 344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.1:33605
W20260812 01:09:40.813176 644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:40.813511 644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:40.813555 644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:40.814980 644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:40.815063 644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:40.815119 644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:40.816767 644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:40.817131 644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:40.817530 644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:40.824359 655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.824350 656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.824601 663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:40.825440 644 server_base.cc:1034] running on GCE node
I20260812 01:09:40.825737 644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:40.826071 644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:40.827279 644 hybrid_clock.cc:648] HybridClock initialized: now 1786496980827251 us; error 52 us; skew 500 ppm
I20260812 01:09:40.829202 644 webserver.cc:460] Webserver started at http://127.17.130.2:33657/ using document root <none> and password file <none>
I20260812 01:09:40.829502 644 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:40.829586 644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:40.829788 644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:40.831113 644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "c4d388fe91924eda90e4538f16a71033"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "a0bfc0fa3035e6af26cf647e0b496142"
server_key_iv: "ed2a249bcc6f489f93891f44e4cf0ec7"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.831549 644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "c4d388fe91924eda90e4538f16a71033"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "a0bfc0fa3035e6af26cf647e0b496142"
server_key_iv: "ed2a249bcc6f489f93891f44e4cf0ec7"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.833247 644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 01:09:40.834317 673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.834537 644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:40.834661 644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "c4d388fe91924eda90e4538f16a71033"
format_stamp: "Formatted at 2026-08-12 01:09:40 on dist-test-slave-tfpf"
server_key: "a0bfc0fa3035e6af26cf647e0b496142"
server_key_iv: "ed2a249bcc6f489f93891f44e4cf0ec7"
server_key_version: "encryptionkey@0"
I20260812 01:09:40.834753 644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:40.852432 644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:40.852855 644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:40.853053 644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:40.853384 644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:40.853787 644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:40.853850 644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.853899 644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:40.853956 644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:40.867311 644 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:34225
I20260812 01:09:40.867411 856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:34225 every 8 connection(s)
I20260812 01:09:40.867851 644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:40.873829 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.868848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:45441 (local address 127.17.130.62:38113)
0812 01:09:40.868953 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:40.868956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:40.869211 (+ 255us) server_negotiation.cc:407] Connection header received
0812 01:09:40.869791 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:40.869793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:40.869805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:40.869861 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:40.870607 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.871147 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.872199 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.872405 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.872498 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.873517 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:40.873519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:40.873519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:40.873525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:40.873542 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:40.873546 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:40.873600 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:40.873708 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:09:40.873741 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 01:09:40.874068 864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:40.868814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:38113 (local address 127.17.130.2:45441)
0812 01:09:40.869053 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:40.869113 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:40.869676 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:40.869951 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:40.869955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:40.869964 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:40.870495 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.870499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.871265 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.871266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.872091 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:40.872094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:40.872510 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:40.872511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:40.872661 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:40.873347 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:40.873347 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:40.873379 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:40.873407 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:40.873609 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:40.873615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:40.873619 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:40.873709 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 01:09:40.873825 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260812 01:09:40.874761 859 heartbeater.cc:347] Connected to a master server at 127.17.130.62:38113
I20260812 01:09:40.874877 859 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:40.875063 859 heartbeater.cc:511] Master 127.17.130.62:38113 requested a full tablet report, sending...
I20260812 01:09:40.875528 344 ts_manager.cc:194] Registered new tserver with Master: c4d388fe91924eda90e4538f16a71033 (127.17.130.2:34225)
I20260812 01:09:40.875618 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 644
I20260812 01:09:40.875732 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:40.875957 17928 external_mini_cluster.cc:1428] Setting key 8a95ead01a1fcc850ce54e5421634b68
I20260812 01:09:40.876271 344 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.2:45441
I20260812 01:09:40.878194 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:38113
--builtin_ntp_servers=127.17.130.20:40991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 01:09:40.987531 868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:40.988113 868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:40.988209 868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:40.990451 868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:40.990513 868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:40.990545 868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:40.992254 868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:40.992514 868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:40.992826 868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:40.999387 881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.999544 875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:40.999387 879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:40.999887 868 server_base.cc:1034] running on GCE node
I20260812 01:09:41.000209 868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:41.000499 868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:41.001713 868 hybrid_clock.cc:648] HybridClock initialized: now 1786496981001674 us; error 58 us; skew 500 ppm
I20260812 01:09:41.003432 868 webserver.cc:460] Webserver started at http://127.17.130.3:44549/ using document root <none> and password file <none>
I20260812 01:09:41.003749 868 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:41.003836 868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:41.004031 868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:41.005435 868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "14dd6cd5e64f4923a61fa409f982b99d"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "238c338ee83ab3b694d812b0ff44ccf9"
server_key_iv: "7ad5ce96d740600e3993e4fcf34bf736"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.006150 868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "14dd6cd5e64f4923a61fa409f982b99d"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "238c338ee83ab3b694d812b0ff44ccf9"
server_key_iv: "7ad5ce96d740600e3993e4fcf34bf736"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.008360 868 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:09:41.009778 888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.010106 868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:41.010243 868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "14dd6cd5e64f4923a61fa409f982b99d"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "238c338ee83ab3b694d812b0ff44ccf9"
server_key_iv: "7ad5ce96d740600e3993e4fcf34bf736"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.010344 868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:41.021766 868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:41.022149 868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:41.022344 868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:41.022742 868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:41.023464 868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:41.023550 868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.023605 868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:41.023658 868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.032155 868 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:37509
I20260812 01:09:41.032202 1088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:37509 every 8 connection(s)
I20260812 01:09:41.032588 868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:41.037237 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 868
I20260812 01:09:41.037685 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:41.038180 17928 external_mini_cluster.cc:1428] Setting key 09a619a4c210999cbef2389ad56ee6d3
I20260812 01:09:41.040261 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.033573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:52101 (local address 127.17.130.62:38113)
0812 01:09:41.033653 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.033655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.033841 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:09:41.034387 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.034388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.034400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.034481 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.035238 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:41.035763 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:41.037029 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:41.037684 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:41.038018 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:41.039792 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:41.039794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:41.039794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:41.039801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:41.039840 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:41.039849 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:41.039909 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:41.040158 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 01:09:41.040194 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 01:09:41.040670 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.033495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:38113 (local address 127.17.130.3:52101)
0812 01:09:41.033706 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:41.033750 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:41.034284 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:41.034577 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:41.034582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:41.034593 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:41.035146 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:41.035151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:41.035880 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:41.035881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:41.036788 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:41.036791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:41.038189 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:41.038192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:41.038496 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:41.039455 (+ 959us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:41.039456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:41.039615 (+ 159us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:41.039664 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:41.039989 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:41.040002 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:41.040007 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:41.040125 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 01:09:41.040271 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260812 01:09:41.041857 1089 heartbeater.cc:347] Connected to a master server at 127.17.130.62:38113
I20260812 01:09:41.041993 1089 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:41.042307 1089 heartbeater.cc:511] Master 127.17.130.62:38113 requested a full tablet report, sending...
I20260812 01:09:41.042903 345 ts_manager.cc:194] Registered new tserver with Master: 14dd6cd5e64f4923a61fa409f982b99d (127.17.130.3:37509)
I20260812 01:09:41.043469 345 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.3:52101
I20260812 01:09:41.051357 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 01:09:41.053542 401 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:41.053018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36460 (local address 127.17.130.62:38113)
0812 01:09:41.053110 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.053112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.053160 (+ 48us) server_negotiation.cc:407] Connection header received
0812 01:09:41.053229 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.053230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.053243 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.053309 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.053438 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36460: BlockingRecv error: recv got EOF from 127.0.0.1:36460 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 01:09:41.053630 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 440
I20260812 01:09:41.075076 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 644
I20260812 01:09:41.086858 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 868
I20260812 01:09:41.100509 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 32758
2026-08-12T01:09:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (778 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T01:09:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:41Z Disabled control of system clock
I20260812 01:09:41.138936 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33177
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:33177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 01:09:41.234962 1112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:41.235404 1112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:41.235450 1112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:41.237548 1112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:41.237646 1112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:41.237671 1112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:41.237700 1112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:41.237751 1112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:41.240576 1112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:33177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:41.240988 1112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:41.241384 1112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:41.247838 1119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.247859 1118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.247910 1122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:41.248651 1112 server_base.cc:1034] running on GCE node
I20260812 01:09:41.248966 1112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:41.249388 1112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:41.250577 1112 hybrid_clock.cc:648] HybridClock initialized: now 1786496981250554 us; error 38 us; skew 500 ppm
I20260812 01:09:41.252295 1112 webserver.cc:460] Webserver started at http://127.17.130.62:43273/ using document root <none> and password file <none>
I20260812 01:09:41.252593 1112 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:41.252671 1112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:41.252832 1112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:41.254089 1112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "2eddbca98af44287be56ac4e4079444f"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "cdb130b086277681af0ca0d0d94e091b"
server_key_iv: "33f1a1c3cdded897e3578a36604fb228"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.254488 1112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "2eddbca98af44287be56ac4e4079444f"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "cdb130b086277681af0ca0d0d94e091b"
server_key_iv: "33f1a1c3cdded897e3578a36604fb228"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.256253 1112 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:09:41.257880 1137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.258353 1112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:41.258541 1112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "2eddbca98af44287be56ac4e4079444f"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "cdb130b086277681af0ca0d0d94e091b"
server_key_iv: "33f1a1c3cdded897e3578a36604fb228"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.258664 1112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:41.263263 1112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:41.263619 1112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:41.263784 1112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:41.268383 1112 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:33177
I20260812 01:09:41.268453 1220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:33177 every 8 connection(s)
I20260812 01:09:41.268878 1112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:41.269829 1221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:41.273370 1221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f: Bootstrap starting.
I20260812 01:09:41.274170 1221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:41.274542 1221 log.cc:824] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:41.275425 1221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f: No bootstrap required, opened a new log
I20260812 01:09:41.275796 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 1112
I20260812 01:09:41.276022 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:41.276283 17928 external_mini_cluster.cc:1428] Setting key e79b1a9aac0d5cab85268afaf3642331
I20260812 01:09:41.278147 1221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eddbca98af44287be56ac4e4079444f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33177 } }
I20260812 01:09:41.278348 1221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:41.278388 1221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eddbca98af44287be56ac4e4079444f, State: Initialized, Role: FOLLOWER
I20260812 01:09:41.278573 1221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eddbca98af44287be56ac4e4079444f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33177 } }
I20260812 01:09:41.278663 1221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:41.278708 1221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:41.278774 1221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:41.279558 1221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eddbca98af44287be56ac4e4079444f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33177 } }
I20260812 01:09:41.279700 1221 leader_election.cc:302] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eddbca98af44287be56ac4e4079444f; no voters:
I20260812 01:09:41.279897 1221 leader_election.cc:288] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:41.279954 1230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:41.280146 1230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [term 1 LEADER]: Becoming Leader. State: Replica: 2eddbca98af44287be56ac4e4079444f, State: Running, Role: LEADER
I20260812 01:09:41.280197 1221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:41.280268 1230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eddbca98af44287be56ac4e4079444f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33177 } }
I20260812 01:09:41.280560 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.277327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.17.130.62:33177)
0812 01:09:41.277584 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.277594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.277636 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:09:41.278091 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.278100 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.278167 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.278324 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.279356 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:41.279360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:41.279360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:41.279365 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:41.279419 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:41.279435 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:41.279536 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:41.279669 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 01:09:41.279845 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260812 01:09:41.280643 1232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eddbca98af44287be56ac4e4079444f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eddbca98af44287be56ac4e4079444f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33177 } } }
I20260812 01:09:41.280725 1232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:41.280901 1233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eddbca98af44287be56ac4e4079444f. Latest consensus state: current_term: 1 leader_uuid: "2eddbca98af44287be56ac4e4079444f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eddbca98af44287be56ac4e4079444f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33177 } } }
I20260812 01:09:41.280936 1237 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:41.280973 1233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:41.281497 1237 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:41.286566 1237 catalog_manager.cc:1347] Generated new cluster ID: b1be49f2edd441cea8594ba3a0759cd4
I20260812 01:09:41.286653 1237 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:41.312453 1237 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:41.312668 1237 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:41.315452 1237 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eddbca98af44287be56ac4e4079444f: Generated new TSK 0
I20260812 01:09:41.315653 1237 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:09:41.321055 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:33177
--builtin_ntp_servers=127.17.130.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 01:09:41.430099 1257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:41.430357 1257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:41.430385 1257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:41.431816 1257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:41.431874 1257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:41.431907 1257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:41.433528 1257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:33177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:41.433842 1257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:41.434211 1257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:41.440361 1264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.440419 1268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.440372 1263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:41.440608 1257 server_base.cc:1034] running on GCE node
I20260812 01:09:41.440901 1257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:41.441211 1257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:41.442404 1257 hybrid_clock.cc:648] HybridClock initialized: now 1786496981442391 us; error 32 us; skew 500 ppm
I20260812 01:09:41.444141 1257 webserver.cc:460] Webserver started at http://127.17.130.1:39629/ using document root <none> and password file <none>
I20260812 01:09:41.444424 1257 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:41.444507 1257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:41.444671 1257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:41.445919 1257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "30278e4f56b144eaa58573c3e43271a0"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "ef6a8400ccfafc00ba0a0ac4407c27b6"
server_key_iv: "fb75d9b7626497b4582d673c5c51791a"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.446316 1257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "30278e4f56b144eaa58573c3e43271a0"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "ef6a8400ccfafc00ba0a0ac4407c27b6"
server_key_iv: "fb75d9b7626497b4582d673c5c51791a"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.447813 1257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:41.448899 1279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.449195 1257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:41.449335 1257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "30278e4f56b144eaa58573c3e43271a0"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "ef6a8400ccfafc00ba0a0ac4407c27b6"
server_key_iv: "fb75d9b7626497b4582d673c5c51791a"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.449427 1257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:41.453425 1257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:41.453754 1257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:41.453927 1257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:41.454272 1257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:41.454707 1257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:41.454758 1257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.454795 1257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:41.454826 1257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.463814 1257 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:43671
I20260812 01:09:41.463938 1456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:43671 every 8 connection(s)
I20260812 01:09:41.464450 1257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:09:41.468868 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 1257
I20260812 01:09:41.469051 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:41.469426 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.465756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:58235 (local address 127.17.130.62:33177)
0812 01:09:41.465862 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.465864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.466150 (+ 286us) server_negotiation.cc:407] Connection header received
0812 01:09:41.466674 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.466675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.466685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.466767 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.468645 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:41.468648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:41.468649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:41.468658 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:41.468769 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:41.468779 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:41.468902 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:41.469303 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 01:09:41.469333 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 01:09:41.469465 17928 external_mini_cluster.cc:1428] Setting key c540ae2ae6d0d62a902020ee6a560d9c
I20260812 01:09:41.469638 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.465707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:33177 (local address 127.17.130.1:58235)
0812 01:09:41.465946 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:41.466002 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:41.466546 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:41.466904 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:41.466909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:41.466918 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:41.468354 (+ 1436us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:41.468355 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:41.468393 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:41.468428 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:41.469020 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:41.469031 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:41.469036 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:41.469235 (+ 199us) client_negotiation.cc:240] Negotiation successful
0812 01:09:41.469339 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
I20260812 01:09:41.470309 1458 heartbeater.cc:347] Connected to a master server at 127.17.130.62:33177
I20260812 01:09:41.470468 1458 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:41.470680 1458 heartbeater.cc:511] Master 127.17.130.62:33177 requested a full tablet report, sending...
I20260812 01:09:41.470793 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:33177
--builtin_ntp_servers=127.17.130.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 01:09:41.471565 1165 ts_manager.cc:194] Registered new tserver with Master: 30278e4f56b144eaa58573c3e43271a0 (127.17.130.1:43671)
W20260812 01:09:41.571676 1463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:41.571997 1463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:41.572045 1463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:41.573619 1463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:41.573691 1463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:41.573745 1463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:41.575374 1463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:33177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:41.575712 1463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:41.576198 1463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:41.582540 1474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:41.582620 1463 server_base.cc:1034] running on GCE node
W20260812 01:09:41.582543 1477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.582623 1470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:41.583026 1463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:41.583308 1463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:41.584477 1463 hybrid_clock.cc:648] HybridClock initialized: now 1786496981584505 us; error 79 us; skew 500 ppm
I20260812 01:09:41.586051 1463 webserver.cc:460] Webserver started at http://127.17.130.2:41011/ using document root <none> and password file <none>
I20260812 01:09:41.586324 1463 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:41.586402 1463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:41.586560 1463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:41.587733 1463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "8ef374a0d54c4c19ad7c7952f509944b"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "79c2e86f076d811034dba5b16113c8a5"
server_key_iv: "a65f7d3bdaedda60d8280bca40cd441c"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.588127 1463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "8ef374a0d54c4c19ad7c7952f509944b"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "79c2e86f076d811034dba5b16113c8a5"
server_key_iv: "a65f7d3bdaedda60d8280bca40cd441c"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.589752 1463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:41.590770 1486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.591056 1463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:41.591185 1463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "8ef374a0d54c4c19ad7c7952f509944b"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "79c2e86f076d811034dba5b16113c8a5"
server_key_iv: "a65f7d3bdaedda60d8280bca40cd441c"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.591266 1463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:41.594990 1463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:41.595292 1463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:41.595439 1463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:41.595666 1463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:41.596017 1463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:41.596079 1463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.596127 1463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:41.596153 1463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.603662 1463 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:42101
I20260812 01:09:41.603744 1671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:42101 every 8 connection(s)
I20260812 01:09:41.604132 1463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:41.607893 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.605261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:45783 (local address 127.17.130.62:33177)
0812 01:09:41.605332 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.605335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.605675 (+ 340us) server_negotiation.cc:407] Connection header received
0812 01:09:41.606247 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.606248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.606255 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.606317 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.607592 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:41.607594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:41.607594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:41.607600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:41.607619 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:41.607624 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:41.607700 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:41.607795 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 01:09:41.607813 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:09:41.608150 1677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.605291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:33177 (local address 127.17.130.2:45783)
0812 01:09:41.605524 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:41.605565 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:41.606132 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:41.606395 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:41.606399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:41.606407 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:41.607424 (+ 1017us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:41.607425 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:41.607457 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:41.607491 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:41.607697 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:41.607704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:41.607708 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:41.607807 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 01:09:41.607884 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":96,"threads_started":1}
I20260812 01:09:41.608760 1673 heartbeater.cc:347] Connected to a master server at 127.17.130.62:33177
I20260812 01:09:41.608867 1673 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:41.609006 1673 heartbeater.cc:511] Master 127.17.130.62:33177 requested a full tablet report, sending...
I20260812 01:09:41.609308 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 1463
I20260812 01:09:41.609429 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:41.609543 1165 ts_manager.cc:194] Registered new tserver with Master: 8ef374a0d54c4c19ad7c7952f509944b (127.17.130.2:42101)
I20260812 01:09:41.609683 17928 external_mini_cluster.cc:1428] Setting key 53e8c2452d47ab3a1ef18f9b4b39e28f
I20260812 01:09:41.610888 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:33177
--builtin_ntp_servers=127.17.130.20:35047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 01:09:41.713587 1678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:41.713817 1678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:41.713841 1678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:41.715027 1678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:41.715075 1678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:41.715102 1678 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:41.716768 1678 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:33177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:41.717213 1678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:41.717571 1678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:41.723232 1686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.723239 1685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:41.723259 1693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:41.723490 1678 server_base.cc:1034] running on GCE node
I20260812 01:09:41.723834 1678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:41.724107 1678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:41.725301 1678 hybrid_clock.cc:648] HybridClock initialized: now 1786496981725284 us; error 34 us; skew 500 ppm
I20260812 01:09:41.727185 1678 webserver.cc:460] Webserver started at http://127.17.130.3:42087/ using document root <none> and password file <none>
I20260812 01:09:41.727502 1678 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:41.727566 1678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:41.727720 1678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:41.729034 1678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "cedfa469a3df43d3b36a6f630bebaa16"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "26112073a129899ef1d4bc107d231fe8"
server_key_iv: "deecb4beec8a3685ca9b07431903f3f6"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.729611 1678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "cedfa469a3df43d3b36a6f630bebaa16"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "26112073a129899ef1d4bc107d231fe8"
server_key_iv: "deecb4beec8a3685ca9b07431903f3f6"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.731171 1678 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:09:41.732002 1705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.732215 1678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:41.732314 1678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "cedfa469a3df43d3b36a6f630bebaa16"
format_stamp: "Formatted at 2026-08-12 01:09:41 on dist-test-slave-tfpf"
server_key: "26112073a129899ef1d4bc107d231fe8"
server_key_iv: "deecb4beec8a3685ca9b07431903f3f6"
server_key_version: "encryptionkey@0"
I20260812 01:09:41.732378 1678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:41.735992 1678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:41.736243 1678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:41.736344 1678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:41.736498 1678 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:41.736792 1678 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:41.736824 1678 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.736845 1678 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:41.736857 1678 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:41.742690 1678 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:38347
I20260812 01:09:41.742808 1890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:38347 every 8 connection(s)
I20260812 01:09:41.743131 1678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:09:41.746549 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.743981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:47453 (local address 127.17.130.62:33177)
0812 01:09:41.744154 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.744157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.744351 (+ 194us) server_negotiation.cc:407] Connection header received
0812 01:09:41.744893 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.744895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.744902 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.744972 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.746269 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:41.746270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:41.746271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:09:41.746275 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:41.746291 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:09:41.746295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:41.746350 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:41.746455 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 01:09:41.746472 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:09:41.746759 1893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:41.743898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:33177 (local address 127.17.130.3:47453)
0812 01:09:41.744205 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:41.744256 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:41.744778 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:41.745047 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:41.745051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:41.745096 (+ 45us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:41.746099 (+ 1003us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:09:41.746100 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:41.746129 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:09:41.746158 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:41.746356 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:41.746362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:41.746365 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:41.746467 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 01:09:41.746532 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260812 01:09:41.747399 1891 heartbeater.cc:347] Connected to a master server at 127.17.130.62:33177
I20260812 01:09:41.747502 1891 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:41.747634 1891 heartbeater.cc:511] Master 127.17.130.62:33177 requested a full tablet report, sending...
I20260812 01:09:41.748026 1165 ts_manager.cc:194] Registered new tserver with Master: cedfa469a3df43d3b36a6f630bebaa16 (127.17.130.3:38347)
I20260812 01:09:41.748642 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 1678
I20260812 01:09:41.748750 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:41.748940 17928 external_mini_cluster.cc:1428] Setting key 0c3b0a598b03a3b4dbfe963a570935c2
I20260812 01:09:41.750429 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 01:09:41.752867 1229 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:09:41.752083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.17.130.62:33177)
0812 01:09:41.752183 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:41.752185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:41.752303 (+ 118us) server_negotiation.cc:407] Connection header received
0812 01:09:41.752334 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:41.752334 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:41.752340 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:41.752398 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:41.752813 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47410: BlockingRecv error: recv got EOF from 127.0.0.1:47410 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 01:09:41.753264 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 1257
I20260812 01:09:41.767043 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 1463
I20260812 01:09:41.788987 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 1678
I20260812 01:09:41.805555 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 1112
2026-08-12T01:09:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (706 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:41 dist-test-slave-tfpf krb5kdc[1910](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:41 dist-test-slave-tfpf krb5kdc[1910](info): set up 2 sockets
Aug 12 01:09:41 dist-test-slave-tfpf krb5kdc[1910](info): commencing operation
krb5kdc: starting...
W20260812 01:09:43.196628 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:09:43.345479 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43535
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:43535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:43.441898 1935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:43.442140 1935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:43.442162 1935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:43.443261 1935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:09:43.443297 1935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:43.443312 1935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:43.443324 1935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:09:43.443337 1935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:09:43.444826 1935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:43535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:43.445044 1935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:43.445323 1935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:43.450615 1941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:43.450647 1943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:43.450609 1947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:43.450801 1935 server_base.cc:1034] running on GCE node
I20260812 01:09:43.451181 1935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:43.451508 1935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:43.452710 1935 hybrid_clock.cc:648] HybridClock initialized: now 1786496983452701 us; error 56 us; skew 500 ppm
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:43.455559 1935 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:09:43.456075 1935 webserver.cc:460] Webserver started at http://127.17.130.62:39825/ using document root <none> and password file <none>
I20260812 01:09:43.456321 1935 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:43.456393 1935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:43.456560 1935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:43.457832 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "9f2ef6787b704b7197df34635e15c521"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "a9acff965218d1d280419d3dd2eb041b"
server_key_iv: "547de475c19029d6a7100e2ea873ec33"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.458240 1935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "9f2ef6787b704b7197df34635e15c521"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "a9acff965218d1d280419d3dd2eb041b"
server_key_iv: "547de475c19029d6a7100e2ea873ec33"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.459725 1935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:09:43.460649 1957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.460906 1935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:43.461025 1935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "9f2ef6787b704b7197df34635e15c521"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "a9acff965218d1d280419d3dd2eb041b"
server_key_iv: "547de475c19029d6a7100e2ea873ec33"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.461181 1935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:43.492308 1935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:43.495110 1935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:43.495304 1935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:43.500731 1935 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:43535
I20260812 01:09:43.500766 2034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:43535 every 8 connection(s)
I20260812 01:09:43.501209 1935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:09:43.502386 2035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:43.504141 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 1935
I20260812 01:09:43.504266 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:09:43.504482 17928 external_mini_cluster.cc:1428] Setting key 8386d5bc7832fbf8aa6bb717f8c12e31
I20260812 01:09:43.505466 2035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521: Bootstrap starting.
I20260812 01:09:43.506598 2035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:43.506978 2035 log.cc:824] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:43.507758 2035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521: No bootstrap required, opened a new log
I20260812 01:09:43.510387 2035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2ef6787b704b7197df34635e15c521" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43535 } }
I20260812 01:09:43.510598 2035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:43.510641 2035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2ef6787b704b7197df34635e15c521, State: Initialized, Role: FOLLOWER
I20260812 01:09:43.510784 2035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2ef6787b704b7197df34635e15c521" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43535 } }
I20260812 01:09:43.510902 2035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:09:43.510941 2035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:09:43.510993 2035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:43.511780 2035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2ef6787b704b7197df34635e15c521" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43535 } }
I20260812 01:09:43.511878 2035 leader_election.cc:302] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f2ef6787b704b7197df34635e15c521; no voters:
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:43.512084 2035 leader_election.cc:288] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:09:43.512108 2042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:43.512233 2042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2ef6787b704b7197df34635e15c521, State: Running, Role: LEADER
I20260812 01:09:43.512336 2042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2ef6787b704b7197df34635e15c521" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43535 } }
I20260812 01:09:43.512409 2035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:09:43.512673 2044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f2ef6787b704b7197df34635e15c521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2ef6787b704b7197df34635e15c521" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43535 } } }
I20260812 01:09:43.512770 2044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:43.512971 2047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f2ef6787b704b7197df34635e15c521. Latest consensus state: current_term: 1 leader_uuid: "9f2ef6787b704b7197df34635e15c521" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2ef6787b704b7197df34635e15c521" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43535 } } }
I20260812 01:09:43.513044 2047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521 [sys.catalog]: This master's current role is: LEADER
I20260812 01:09:43.513242 2050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:09:43.513803 2050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:09:43.516158 2050 catalog_manager.cc:1347] Generated new cluster ID: 4b566dca336841c4bb8cf64343610088
I20260812 01:09:43.516213 2050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:09:43.517261 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:43.505120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44512 (local address 127.17.130.62:43535)
0812 01:09:43.505312 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:43.505323 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:43.505359 (+ 36us) server_negotiation.cc:407] Connection header received
0812 01:09:43.505829 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:43.505837 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:43.505886 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:43.506010 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:43.506973 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.508028 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.509175 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.509585 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.509677 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:43.512721 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:43.512725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:43.512725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:43.512739 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:43.515053 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:43.515556 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:43.515559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:43.515560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:43.515658 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:43.515896 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:43.515897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:43.515898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:43.516231 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:43.516389 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:43.516863 (+ 474us) server_negotiation.cc:299] Negotiation successful
0812 01:09:43.517005 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260812 01:09:43.537249 2050 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:09:43.538034 2050 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:09:43.542547 2050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521: Generated new TSK 0
I20260812 01:09:43.542798 2050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:09:43.577251 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43535
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:43.673477 2077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:43.673732 2077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:43.673770 2077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:43.675127 2077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:43.675222 2077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:43.675274 2077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:09:43.676928 2077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:43.677426 2077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:43.677716 2077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:43.683400 2084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:43.683418 2088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:43.683465 2085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:43.683676 2077 server_base.cc:1034] running on GCE node
I20260812 01:09:43.683955 2077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:43.684250 2077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:43.685417 2077 hybrid_clock.cc:648] HybridClock initialized: now 1786496983685393 us; error 37 us; skew 500 ppm
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:43.688233 2077 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:09:43.688905 2077 webserver.cc:460] Webserver started at http://127.17.130.1:35727/ using document root <none> and password file <none>
I20260812 01:09:43.689235 2077 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:43.689289 2077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:43.689432 2077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:43.690543 2077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "ba541e655b0d4b6d9f501868b88f8739"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "096dc05316d63676d61ccc9d6a824b28"
server_key_iv: "15945c524522ed05cef5c132027554f4"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.690927 2077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "ba541e655b0d4b6d9f501868b88f8739"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "096dc05316d63676d61ccc9d6a824b28"
server_key_iv: "15945c524522ed05cef5c132027554f4"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.692373 2077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:09:43.693362 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.693555 2077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:43.693653 2077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "ba541e655b0d4b6d9f501868b88f8739"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "096dc05316d63676d61ccc9d6a824b28"
server_key_iv: "15945c524522ed05cef5c132027554f4"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.693712 2077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:43.705307 2077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:43.707623 2077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:43.707748 2077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:43.707965 2077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:43.708285 2077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:43.708323 2077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.708345 2077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:43.708357 2077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.714859 2077 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:34745
I20260812 01:09:43.714941 2269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:34745 every 8 connection(s)
I20260812 01:09:43.715355 2077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:43.725615 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 2077
I20260812 01:09:43.725735 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:09:43.726060 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:43.716355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:51633 (local address 127.17.130.62:43535)
0812 01:09:43.716445 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:43.716448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:43.716686 (+ 238us) server_negotiation.cc:407] Connection header received
0812 01:09:43.717361 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:43.717363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:43.717441 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:43.717527 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:43.718584 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.719132 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.720157 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.720362 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.720426 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:43.722819 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:43.722821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:43.722821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:43.722829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:43.724436 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:43.724879 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:43.724880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:43.724881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:43.724952 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:43.725227 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:43.725228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:43.725228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:43.725345 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:43.725404 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:43.725898 (+ 494us) server_negotiation.cc:299] Negotiation successful
0812 01:09:43.725989 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 01:09:43.726073 17928 external_mini_cluster.cc:1428] Setting key 2347ea793cfc1c5cfc36e6b740a86102
I20260812 01:09:43.726428 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:43.716280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43535 (local address 127.17.130.1:51633)
0812 01:09:43.716514 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:43.716566 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:43.717235 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:43.717642 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:43.717645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:43.718267 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:43.718469 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:43.718474 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.719246 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.719247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:43.720056 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:43.720058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.720466 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.720467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:43.720600 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:43.721192 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:43.721201 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:43.722690 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:43.724542 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:43.724549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:43.724552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:43.724790 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:43.725027 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:43.725027 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:43.725028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:43.725164 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:43.725439 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:43.725455 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:43.725667 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 01:09:43.725853 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 01:09:43.726155 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":122,"threads_started":1}
I20260812 01:09:43.726963 2270 heartbeater.cc:347] Connected to a master server at 127.17.130.62:43535
I20260812 01:09:43.727077 2270 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:43.727314 2270 heartbeater.cc:511] Master 127.17.130.62:43535 requested a full tablet report, sending...
I20260812 01:09:43.727831 1983 ts_manager.cc:194] Registered new tserver with Master: ba541e655b0d4b6d9f501868b88f8739 (127.17.130.1:34745)
I20260812 01:09:43.728456 1983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:51633
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:09:43.757429 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43535
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:43.864892 2276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:43.865293 2276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:43.865348 2276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:43.868086 2276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:43.868160 2276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:43.868196 2276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:09:43.871119 2276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:43.871519 2276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:43.871858 2276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:43.878913 2288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:43.878931 2285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:43.879063 2293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:43.883263 2276 server_base.cc:1034] running on GCE node
I20260812 01:09:43.883569 2276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:43.883877 2276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:43.885226 2276 hybrid_clock.cc:648] HybridClock initialized: now 1786496983885181 us; error 51 us; skew 500 ppm
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:43.888823 2276 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:09:43.889554 2276 webserver.cc:460] Webserver started at http://127.17.130.2:40973/ using document root <none> and password file <none>
I20260812 01:09:43.889886 2276 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:43.889966 2276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:43.890153 2276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:43.891913 2276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "82ff10c7a44b4fb4ae67476bb4fe7708"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "2f4073344b586380ad5bb84615b6a9a4"
server_key_iv: "14041030a98e5b1fedd9b9915a149612"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.892452 2276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "82ff10c7a44b4fb4ae67476bb4fe7708"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "2f4073344b586380ad5bb84615b6a9a4"
server_key_iv: "14041030a98e5b1fedd9b9915a149612"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.894634 2276 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 01:09:43.895853 2305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.896166 2276 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:09:43.896292 2276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "82ff10c7a44b4fb4ae67476bb4fe7708"
format_stamp: "Formatted at 2026-08-12 01:09:43 on dist-test-slave-tfpf"
server_key: "2f4073344b586380ad5bb84615b6a9a4"
server_key_iv: "14041030a98e5b1fedd9b9915a149612"
server_key_version: "encryptionkey@0"
I20260812 01:09:43.896402 2276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:43.923666 2276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:43.928262 2276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:43.928478 2276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:43.928836 2276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:43.929328 2276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:43.929392 2276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.929432 2276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:43.929456 2276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:43.938591 2276 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:43261
I20260812 01:09:43.938833 2484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:43261 every 8 connection(s)
I20260812 01:09:43.939205 2276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:09:43.946285 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 2276
I20260812 01:09:43.946425 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:09:43.946676 17928 external_mini_cluster.cc:1428] Setting key 056a591e617249aa8771926c3f9c838e
Aug 12 01:09:43 dist-test-slave-tfpf krb5kdc[1910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496983, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:43.955523 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:43.940423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:42125 (local address 127.17.130.62:43535)
0812 01:09:43.940515 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:43.940517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:43.940671 (+ 154us) server_negotiation.cc:407] Connection header received
0812 01:09:43.941266 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:43.941268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:43.941284 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:43.941368 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:43.942896 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.943902 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.945391 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.945745 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.946099 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:43.951222 (+ 5123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:43.951225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:43.951226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:43.951239 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:43.953382 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:43.954134 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:43.954136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:43.954137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:43.954207 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:43.954546 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:43.954548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:43.954548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:43.954730 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:43.954794 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:43.955282 (+ 488us) server_negotiation.cc:299] Negotiation successful
0812 01:09:43.955398 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 01:09:43.956614 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:43.940228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43535 (local address 127.17.130.2:42125)
0812 01:09:43.940532 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:43.940582 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:43.941148 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:43.941455 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:43.941459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:43.942262 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:43.942620 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:43.942636 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.944049 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.944050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:43.945285 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:43.945289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:43.946104 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:43.946105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:43.946311 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:43.947199 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:43.947210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:43.951082 (+ 3872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:43.953548 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:43.953562 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:43.953566 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:43.953999 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:43.954289 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:43.954290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:43.954290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:43.954455 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:43.954834 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:43.954843 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:43.955081 (+ 238us) client_negotiation.cc:769] Sending connection context
0812 01:09:43.955982 (+ 901us) client_negotiation.cc:240] Negotiation successful
0812 01:09:43.956274 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:09:43.957751 2485 heartbeater.cc:347] Connected to a master server at 127.17.130.62:43535
I20260812 01:09:43.957893 2485 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:43.958272 2485 heartbeater.cc:511] Master 127.17.130.62:43535 requested a full tablet report, sending...
I20260812 01:09:43.958807 1983 ts_manager.cc:194] Registered new tserver with Master: 82ff10c7a44b4fb4ae67476bb4fe7708 (127.17.130.2:43261)
I20260812 01:09:43.959257 1983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:42125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:09:43.981856 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43535
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:09:44.090161 2492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:09:44.090430 2492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:09:44.090463 2492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:09:44.092167 2492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:09:44.092245 2492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:09:44.092278 2492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:09:44.094201 2492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:43535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:09:44.094523 2492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:09:44.094813 2492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:09:44.103806 2500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:09:44.104063 2505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:44.104110 2492 server_base.cc:1034] running on GCE node
W20260812 01:09:44.103806 2501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:09:44.104540 2492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:09:44.104991 2492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:09:44.106259 2492 hybrid_clock.cc:648] HybridClock initialized: now 1786496984106231 us; error 61 us; skew 500 ppm
Aug 12 01:09:44 dist-test-slave-tfpf krb5kdc[1910](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496984, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:09:44.114259 2492 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:09:44.114863 2492 webserver.cc:460] Webserver started at http://127.17.130.3:45039/ using document root <none> and password file <none>
I20260812 01:09:44.115139 2492 fs_manager.cc:359] Metadata directory not provided
I20260812 01:09:44.115214 2492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:09:44.115376 2492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:09:44.116685 2492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "e9a163e5a4854287a4ba1de8eb86fe70"
format_stamp: "Formatted at 2026-08-12 01:09:44 on dist-test-slave-tfpf"
server_key: "e0c231d8d280042e291a89353382c3ea"
server_key_iv: "fb54e1780ad84ae0ead9ae89e4afcab3"
server_key_version: "encryptionkey@0"
I20260812 01:09:44.117228 2492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "e9a163e5a4854287a4ba1de8eb86fe70"
format_stamp: "Formatted at 2026-08-12 01:09:44 on dist-test-slave-tfpf"
server_key: "e0c231d8d280042e291a89353382c3ea"
server_key_iv: "fb54e1780ad84ae0ead9ae89e4afcab3"
server_key_version: "encryptionkey@0"
I20260812 01:09:44.119062 2492 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 01:09:44.120132 2516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:44.120509 2492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:09:44.120633 2492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "e9a163e5a4854287a4ba1de8eb86fe70"
format_stamp: "Formatted at 2026-08-12 01:09:44 on dist-test-slave-tfpf"
server_key: "e0c231d8d280042e291a89353382c3ea"
server_key_iv: "fb54e1780ad84ae0ead9ae89e4afcab3"
server_key_version: "encryptionkey@0"
I20260812 01:09:44.120731 2492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:09:44.137254 2492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:09:44.140589 2492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:09:44.140817 2492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:09:44.141228 2492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:09:44.141680 2492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:09:44.141727 2492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:44.141782 2492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:09:44.141832 2492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:09:44.150656 2492 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:39651
I20260812 01:09:44.150827 2703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:39651 every 8 connection(s)
I20260812 01:09:44.151150 2492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:09:44 dist-test-slave-tfpf krb5kdc[1910](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496984, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:09:44.161369 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 2492
I20260812 01:09:44.161490 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:09:44.161718 17928 external_mini_cluster.cc:1428] Setting key cae81bf2f8aa2e040330a31f19a8e9c0
I20260812 01:09:44.162590 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.152089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:55639 (local address 127.17.130.62:43535)
0812 01:09:44.152160 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.152162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.152240 (+ 78us) server_negotiation.cc:407] Connection header received
0812 01:09:44.152827 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.152829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.152843 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.152919 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:44.154037 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.154540 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.155606 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.155804 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.155862 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.158721 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:44.158723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:44.158723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:44.158731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:44.160324 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:44.160870 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:44.160872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:44.160872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:44.161029 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:44.161657 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:44.161658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:44.161659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:44.161832 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:44.161924 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.162360 (+ 436us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.162483 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 01:09:44.162770 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.151904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43535 (local address 127.17.130.3:55639)
0812 01:09:44.152091 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.152137 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:44.152675 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:44.152967 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:44.152970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:44.153689 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:09:44.153896 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.153900 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.154656 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.154657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.155502 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.155505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.155913 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.155915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.156068 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.156915 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:09:44.156925 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:09:44.158613 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:09:44.160431 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:44.160440 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:44.160443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:44.160700 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:44.161325 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:09:44.161327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:09:44.161327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:09:44.161494 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:09:44.161921 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:09:44.161929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:09:44.162140 (+ 211us) client_negotiation.cc:769] Sending connection context
0812 01:09:44.162287 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 01:09:44.162487 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260812 01:09:44.163345 2704 heartbeater.cc:347] Connected to a master server at 127.17.130.62:43535
I20260812 01:09:44.163472 2704 heartbeater.cc:464] Registering TS with master...
I20260812 01:09:44.163708 2704 heartbeater.cc:511] Master 127.17.130.62:43535 requested a full tablet report, sending...
I20260812 01:09:44.164389 1983 ts_manager.cc:194] Registered new tserver with Master: e9a163e5a4854287a4ba1de8eb86fe70 (127.17.130.3:39651)
I20260812 01:09:44.164912 1983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:55639
I20260812 01:09:44.174141 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:09:44.183599 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.175767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.17.130.62:43535)
0812 01:09:44.175862 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.175864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.175872 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:09:44.175893 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.175894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.175903 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.175970 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:09:44.176584 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.177172 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.177899 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.178099 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.178176 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.179736 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:09:44.179740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:09:44.179740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:09:44.179752 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:09:44.182132 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:44.182624 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:44.182626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:44.182626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:44.182695 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:09:44.182917 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:09:44.182918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:09:44.182918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:09:44.183114 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:09:44.183241 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.183381 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.183483 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 01:09:44.186597 1983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:09:44.188427 1983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:09:44.196393 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.192017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.17.130.3:39651)
0812 01:09:44.192240 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.192244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.192618 (+ 374us) server_negotiation.cc:407] Connection header received
0812 01:09:44.192670 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.192673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.192691 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.192794 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:44.192914 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.193742 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.195368 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.195865 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.195972 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.196039 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.196248 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.196295 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":115,"threads_started":1}
I20260812 01:09:44.196444 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.191645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34745 (local address 127.0.0.1:42996)
0812 01:09:44.192125 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.192138 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:44.192205 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:44.192648 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:44.192649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:44.192651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:44.192687 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.192688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.194046 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.194046 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.195226 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.195229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.196158 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.196159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.196218 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.196221 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:09:44.196251 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 01:09:44.196327 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":62,"thread_start_us":86,"threads_started":1}
I20260812 01:09:44.196507 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.191799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56916 (local address 127.17.130.2:43261)
0812 01:09:44.192035 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.192041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.192301 (+ 260us) server_negotiation.cc:407] Connection header received
0812 01:09:44.192364 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.192366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.192383 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.192467 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:44.192686 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.193928 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.195177 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.195929 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.196006 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.196066 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.196380 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.196424 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":124,"threads_started":1}
I20260812 01:09:44.196435 2723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.191893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39651 (local address 127.0.0.1:39838)
0812 01:09:44.192304 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.192311 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:44.192348 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:44.192791 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:44.192792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:44.192794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:44.192851 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.192853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.193851 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.193852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.195283 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.195285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.195975 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.195976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.196075 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.196098 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 01:09:44.196228 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 01:09:44.196330 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"mutex_wait_us":62,"thread_start_us":91,"threads_started":1}
I20260812 01:09:44.197233 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.191645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:43261 (local address 127.0.0.1:56916)
0812 01:09:44.192050 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.192101 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:44.192203 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:44.192492 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:44.192495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:44.192497 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:44.192583 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.192586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.194079 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.194080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.195071 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.195074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.196106 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.196108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.196176 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.196181 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:44.196229 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 01:09:44.196309 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":75,"threads_started":1}
I20260812 01:09:44.197592 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.191877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.17.130.1:34745)
0812 01:09:44.192465 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.192470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.192480 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:09:44.192523 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.192525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.192540 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.192763 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:44.192848 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.193936 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.195317 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.196020 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.196348 (+ 328us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.196894 (+ 546us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.197068 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.197218 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":107,"threads_started":1}
I20260812 01:09:44.197991 2380 tablet_service.cc:1467] Processing CreateTablet for tablet c81281e265aa4acd9dbe4330b43edbd3 (DEFAULT_TABLE table=test-table [id=2a2fe9739eae4988b78a632b7150b272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:44.197870 2601 tablet_service.cc:1467] Processing CreateTablet for tablet c81281e265aa4acd9dbe4330b43edbd3 (DEFAULT_TABLE table=test-table [id=2a2fe9739eae4988b78a632b7150b272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:44.198446 2380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81281e265aa4acd9dbe4330b43edbd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:44.198630 2601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81281e265aa4acd9dbe4330b43edbd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:44.198629 2169 tablet_service.cc:1467] Processing CreateTablet for tablet c81281e265aa4acd9dbe4330b43edbd3 (DEFAULT_TABLE table=test-table [id=2a2fe9739eae4988b78a632b7150b272]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:09:44.199023 2169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c81281e265aa4acd9dbe4330b43edbd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:09:44.201941 2729 tablet_bootstrap.cc:492] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Bootstrap starting.
I20260812 01:09:44.202816 2729 tablet_bootstrap.cc:654] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:44.203105 2730 tablet_bootstrap.cc:492] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: Bootstrap starting.
I20260812 01:09:44.203485 2729 log.cc:824] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:44.203941 2730 tablet_bootstrap.cc:654] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:44.204267 2730 log.cc:824] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:44.204345 2729 tablet_bootstrap.cc:492] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: No bootstrap required, opened a new log
I20260812 01:09:44.204576 2729 ts_tablet_manager.cc:1397] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:09:44.204998 2733 tablet_bootstrap.cc:492] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Bootstrap starting.
I20260812 01:09:44.205727 2730 tablet_bootstrap.cc:492] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: No bootstrap required, opened a new log
I20260812 01:09:44.205775 2733 tablet_bootstrap.cc:654] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Neither blocks nor log segments found. Creating new log.
I20260812 01:09:44.205807 2730 ts_tablet_manager.cc:1397] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:09:44.206372 2733 log.cc:824] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Log is configured to *not* fsync() on all Append() calls
I20260812 01:09:44.206638 2729 raft_consensus.cc:348] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.206831 2729 raft_consensus.cc:374] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:44.206871 2729 raft_consensus.cc:729] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82ff10c7a44b4fb4ae67476bb4fe7708, State: Initialized, Role: FOLLOWER
I20260812 01:09:44.207022 2729 consensus_queue.cc:260] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.207360 2485 heartbeater.cc:503] Master 127.17.130.62:43535 was elected leader, sending a full tablet report...
I20260812 01:09:44.207368 2729 ts_tablet_manager.cc:1428] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:44.207434 2733 tablet_bootstrap.cc:492] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: No bootstrap required, opened a new log
I20260812 01:09:44.207607 2733 ts_tablet_manager.cc:1397] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:09:44.208339 2730 raft_consensus.cc:348] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.208510 2730 raft_consensus.cc:374] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:44.208549 2730 raft_consensus.cc:729] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba541e655b0d4b6d9f501868b88f8739, State: Initialized, Role: FOLLOWER
I20260812 01:09:44.208678 2730 consensus_queue.cc:260] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.209481 2730 ts_tablet_manager.cc:1428] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 01:09:44.209568 2270 heartbeater.cc:503] Master 127.17.130.62:43535 was elected leader, sending a full tablet report...
I20260812 01:09:44.209831 2733 raft_consensus.cc:348] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.210013 2733 raft_consensus.cc:374] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:09:44.210062 2733 raft_consensus.cc:729] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9a163e5a4854287a4ba1de8eb86fe70, State: Initialized, Role: FOLLOWER
I20260812 01:09:44.210188 2733 consensus_queue.cc:260] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.210520 2733 ts_tablet_manager.cc:1428] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:09:44.210546 2704 heartbeater.cc:503] Master 127.17.130.62:43535 was elected leader, sending a full tablet report...
I20260812 01:09:44.215116 2738 raft_consensus.cc:482] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:09:44.215234 2738 raft_consensus.cc:504] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.215667 2738 leader_election.cc:288] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82ff10c7a44b4fb4ae67476bb4fe7708 (127.17.130.2:43261), e9a163e5a4854287a4ba1de8eb86fe70 (127.17.130.3:39651)
W20260812 01:09:44.216492 2271 tablet.cc:2369] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:09:44.220027 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.215901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:43261 (local address 127.17.130.1:55377)
0812 01:09:44.215968 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.215982 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:44.216027 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:44.216379 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:44.216380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:44.216383 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:44.216489 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.216491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.217371 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.217372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.218645 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.218648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.219710 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.219712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.219800 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.219805 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:09:44.219882 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 01:09:44.219938 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:09:44.220149 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.216118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:55377 (local address 127.17.130.2:43261)
0812 01:09:44.216219 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.216222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.216233 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:44.216265 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.216267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.216282 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.216551 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:44.216630 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.217295 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.218758 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.219611 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.219678 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.219686 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.219960 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.220026 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 01:09:44.220664 2743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.216034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39651 (local address 127.17.130.1:47133)
0812 01:09:44.216407 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.216423 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:09:44.216512 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:09:44.217050 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:09:44.217051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:09:44.217053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:09:44.217155 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.217161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.218227 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.218228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.219390 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:09:44.219393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.220252 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.220253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:09:44.220331 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.220335 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:09:44.220399 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:09:44.220434 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":188,"threads_started":1}
I20260812 01:09:44.220613 2407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81281e265aa4acd9dbe4330b43edbd3" candidate_uuid: "ba541e655b0d4b6d9f501868b88f8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" is_pre_election: true
I20260812 01:09:44.220835 2407 raft_consensus.cc:2454] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba541e655b0d4b6d9f501868b88f8739 in term 0.
I20260812 01:09:44.221210 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.216365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47133 (local address 127.17.130.3:39651)
0812 01:09:44.216828 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.216831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.216842 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:09:44.216870 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.216871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.216882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.216980 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:09:44.217238 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.217944 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.219534 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.220125 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.220207 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.220217 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.221039 (+ 822us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.221127 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419}
I20260812 01:09:44.221535 2105 leader_election.cc:302] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82ff10c7a44b4fb4ae67476bb4fe7708, ba541e655b0d4b6d9f501868b88f8739; no voters:
I20260812 01:09:44.221719 2738 raft_consensus.cc:2781] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:09:44.221830 2738 raft_consensus.cc:482] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:09:44.221889 2738 raft_consensus.cc:3037] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:44.222170 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81281e265aa4acd9dbe4330b43edbd3" candidate_uuid: "ba541e655b0d4b6d9f501868b88f8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" is_pre_election: true
I20260812 01:09:44.222364 2626 raft_consensus.cc:2454] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba541e655b0d4b6d9f501868b88f8739 in term 0.
I20260812 01:09:44.223169 2738 raft_consensus.cc:504] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.223348 2738 leader_election.cc:288] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [CANDIDATE]: Term 1 election: Requested vote from peers 82ff10c7a44b4fb4ae67476bb4fe7708 (127.17.130.2:43261), e9a163e5a4854287a4ba1de8eb86fe70 (127.17.130.3:39651)
I20260812 01:09:44.223603 2626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81281e265aa4acd9dbe4330b43edbd3" candidate_uuid: "ba541e655b0d4b6d9f501868b88f8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9a163e5a4854287a4ba1de8eb86fe70"
I20260812 01:09:44.223639 2407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c81281e265aa4acd9dbe4330b43edbd3" candidate_uuid: "ba541e655b0d4b6d9f501868b88f8739" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708"
I20260812 01:09:44.223719 2407 raft_consensus.cc:3037] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:44.223738 2626 raft_consensus.cc:3037] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:09:44.224639 2626 raft_consensus.cc:2454] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba541e655b0d4b6d9f501868b88f8739 in term 1.
I20260812 01:09:44.224807 2105 leader_election.cc:302] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba541e655b0d4b6d9f501868b88f8739, e9a163e5a4854287a4ba1de8eb86fe70; no voters:
I20260812 01:09:44.224936 2738 raft_consensus.cc:2781] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:09:44.225185 2738 raft_consensus.cc:686] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [term 1 LEADER]: Becoming Leader. State: Replica: ba541e655b0d4b6d9f501868b88f8739, State: Running, Role: LEADER
I20260812 01:09:44.225261 2407 raft_consensus.cc:2454] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba541e655b0d4b6d9f501868b88f8739 in term 1.
I20260812 01:09:44.225327 2738 consensus_queue.cc:237] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } }
I20260812 01:09:44.226341 1983 catalog_manager.cc:5495] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba541e655b0d4b6d9f501868b88f8739 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "ba541e655b0d4b6d9f501868b88f8739" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba541e655b0d4b6d9f501868b88f8739" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34745 } health_report { overall_health: HEALTHY } } }
I20260812 01:09:44.233305 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.230834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42998 (local address 127.17.130.1:34745)
0812 01:09:44.230885 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:09:44.230887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:09:44.230896 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:09:44.230964 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:09:44.230965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:09:44.230984 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:09:44.231042 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:09:44.231163 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.231628 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.232441 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:09:44.232653 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:09:44.232718 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:09:44.232924 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:09:44.233023 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:09:44.233167 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 01:09:44.233208 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 01:09:44.233228 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:09:44.235450 2407 raft_consensus.cc:1261] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 1 FOLLOWER]: Refusing update from remote peer ba541e655b0d4b6d9f501868b88f8739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:44.235498 2626 raft_consensus.cc:1261] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 1 FOLLOWER]: Refusing update from remote peer ba541e655b0d4b6d9f501868b88f8739: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:09:44.235877 2744 consensus_queue.cc:1035] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82ff10c7a44b4fb4ae67476bb4fe7708" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:44.236006 2738 consensus_queue.cc:1035] T c81281e265aa4acd9dbe4330b43edbd3 P ba541e655b0d4b6d9f501868b88f8739 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9a163e5a4854287a4ba1de8eb86fe70" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:09:44.239249 2749 mvcc.cc:205] Tried to move back new op lower bound from 7317491647424966656 to 7317491647387799552. Current Snapshot: MvccSnapshot[applied={T|T < 7317491647424966656}]
I20260812 01:09:44.240017 2750 mvcc.cc:205] Tried to move back new op lower bound from 7317491647424966656 to 7317491647387799552. Current Snapshot: MvccSnapshot[applied={T|T < 7317491647424966656}]
I20260812 01:09:44.241703 1983 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:44.241887 1983 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:09:44.243113 1983 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f2ef6787b704b7197df34635e15c521: Sending DeleteTablet for 3 replicas of tablet c81281e265aa4acd9dbe4330b43edbd3
I20260812 01:09:44.243566 2380 tablet_service.cc:1514] Processing DeleteTablet for tablet c81281e265aa4acd9dbe4330b43edbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:44 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:56916
I20260812 01:09:44.243616 2601 tablet_service.cc:1514] Processing DeleteTablet for tablet c81281e265aa4acd9dbe4330b43edbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:44 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:39838
I20260812 01:09:44.243796 2169 tablet_service.cc:1514] Processing DeleteTablet for tablet c81281e265aa4acd9dbe4330b43edbd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:09:44 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42996
I20260812 01:09:44.243873 2768 tablet_replica.cc:330] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: stopping tablet replica
I20260812 01:09:44.243994 2768 raft_consensus.cc:2229] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:44.244213 2768 raft_consensus.cc:2258] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:44.244376 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 2077
I20260812 01:09:44.253950 2768 ts_tablet_manager.cc:1905] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:44.255555 2768 ts_tablet_manager.cc:1918] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:44.255661 2768 log.cc:1196] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/c81281e265aa4acd9dbe4330b43edbd3
I20260812 01:09:44.255954 2768 ts_tablet_manager.cc:1939] T c81281e265aa4acd9dbe4330b43edbd3 P e9a163e5a4854287a4ba1de8eb86fe70: Deleting consensus metadata
I20260812 01:09:44.256421 1962 catalog_manager.cc:4841] TS e9a163e5a4854287a4ba1de8eb86fe70 (127.17.130.3:39651): tablet c81281e265aa4acd9dbe4330b43edbd3 (table test-table [id=2a2fe9739eae4988b78a632b7150b272]) successfully deleted
I20260812 01:09:44.261274 2767 tablet_replica.cc:330] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: stopping tablet replica
I20260812 01:09:44.261463 2767 raft_consensus.cc:2229] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:09:44.261647 2767 raft_consensus.cc:2258] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:09:44.262147 2767 ts_tablet_manager.cc:1905] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:09:44.263592 2767 ts_tablet_manager.cc:1918] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:09:44.263662 2767 log.cc:1196] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/c81281e265aa4acd9dbe4330b43edbd3
I20260812 01:09:44.263936 2767 ts_tablet_manager.cc:1939] T c81281e265aa4acd9dbe4330b43edbd3 P 82ff10c7a44b4fb4ae67476bb4fe7708: Deleting consensus metadata
W20260812 01:09:44.264228 1960 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:34745 (error 108)
I20260812 01:09:44.264592 1962 catalog_manager.cc:4841] TS 82ff10c7a44b4fb4ae67476bb4fe7708 (127.17.130.2:43261): tablet c81281e265aa4acd9dbe4330b43edbd3 (table test-table [id=2a2fe9739eae4988b78a632b7150b272]) successfully deleted
I20260812 01:09:44.264844 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:09:44.264681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34745 (local address 127.0.0.1:43014)
0812 01:09:44.264740 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:09:44.264778 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 01:09:44.264993 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 2276
W20260812 01:09:44.265043 1960 catalog_manager.cc:4584] TS ba541e655b0d4b6d9f501868b88f8739 (127.17.130.1:34745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c81281e265aa4acd9dbe4330b43edbd3: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34745: connect: Connection refused (error 111)
I20260812 01:09:44.285686 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 2492
I20260812 01:09:44.304762 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 1935
2026-08-12T01:09:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2496 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:09:44 dist-test-slave-tfpf krb5kdc[2774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:09:44 dist-test-slave-tfpf krb5kdc[2774](info): set up 2 sockets
Aug 12 01:09:44 dist-test-slave-tfpf krb5kdc[2774](info): commencing operation
krb5kdc: starting...
W20260812 01:09:45.615703 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:09:45 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786496985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:09:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:09:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/rangeradmin_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/rangeradmin_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/rangerlookup_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/rangerlookup_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
I20260812 01:09:45.816712 17928 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskolrujf/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/postgres -l logfile start
2026-08-12 01:09:48.734 UTC [2815] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 01:09:48.734 UTC [2815] LOG: listening on IPv4 address "127.17.130.20", port 53775
2026-08-12 01:09:48.739 UTC [2815] LOG: listening on Unix socket "/tmp/.s.PGSQL.53775"
2026-08-12 01:09:48.765 UTC [2821] LOG: database system was shut down at 2026-08-12 01:09:47 UTC
2026-08-12 01:09:48.770 UTC [2815] LOG: database system is ready to accept connections
I20260812 01:09:50.044430 17928 mini_postgres.cc:96] Postgres bound to 53775
2026-08-12 01:09:50.054 UTC [2838] FATAL: database "slave" does not exist
127.17.130.20:53775 - accepting connections
I20260812 01:09:50.055073 17928 mini_ranger.cc:161] Starting Ranger...
I20260812 01:09:50.073537 17928 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 01:09:50.734723 17928 mini_ranger.cc:87] Created ranger Postgres database
I20260812 01:09:50.734829 17928 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-admin
2026-08-12 01:09:51,122 [I] DB FLAVOR :POSTGRES
2026-08-12 01:09:51,123 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:09:51,123 [I] Checking connection..
2026-08-12 01:09:51,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 01:09:51,408 [I] Checking connection passed.
2026-08-12 01:09:51,408 [I] --------- Verifying version history table ---------
2026-08-12 01:09:51,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:09:51,701 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:09:51,702 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:09:51,989 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:09:51,989 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 01:09:51,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 01:09:52.260 UTC [2945] WARNING: there is no transaction in progress
2026-08-12 01:09:52,271 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 01:09:52,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:09:52,556 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 01:09:52,557 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:09:52,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 01:09:52,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 01:09:53,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tfpf.c.gcp-upstream.internal','N') ;"
2026-08-12 01:09:53,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:09:53,682 [I] Table x_portal_user does not exist in database ranger
2026-08-12 01:09:53,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:09:53,986 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 01:09:53,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:09:54,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:09:54,552 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 01:09:54,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 01:09:55.724 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.744 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.762 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.779 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.797 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.817 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.879 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.887 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.903 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.914 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.925 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.942 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.953 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.961 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.968 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:55.976 UTC [3118] WARNING: there is no transaction in progress
2026-08-12 01:09:56,805 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 01:09:56,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 01:09:57,075 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 01:09:57,076 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:09:57,351 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 01:09:57,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:09:57,662 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 01:09:57,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:09:57,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:09:58,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tfpf.c.gcp-upstream.internal';"
2026-08-12 01:09:58,504 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 01:09:58,505 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 01:09:58,778 [I] DB_PATCHES have already been applied
I20260812 01:09:58.785264 17928 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-admin:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskolrujf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 01:09:58.785351 17928 mini_ranger.cc:193] Using host: 127.17.130.20
I20260812 01:09:58.790119 17928 mini_ranger.cc:227] Ranger admin URL: http://127.17.130.20:39199
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 1:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 01:10:00.285346 17928 mini_ranger.cc:160] Time spent starting Ranger: real 10.230s user 0.000s sys 0.013s
Aug 12, 2026 1:10:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 1:10:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 1:10:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 1:10:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 1:10:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 01:10:24.257288 17928 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/rangerkms_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/rangerkms_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.130.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.130.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 01:10:24 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497024, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 01:10:24.366786 17928 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 01:10:24.380489 17928 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 01:10:25.734761 17928 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 01:10:25.734884 17928 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms
I20260812 01:10:25.734908 17928 mini_ranger_kms.cc:227] Using postgres at 127.17.130.20:53775
2026-08-12 01:10:26,547 [I] DB FLAVOR :POSTGRES
2026-08-12 01:10:26,548 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:10:26,548 [I] Checking connection
2026-08-12 01:10:26,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 01:10:26,872 [I] connection success
2026-08-12 01:10:26,872 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 01:10:26,872 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 01:10:26,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 01:10:27,218 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 01:10:27,218 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:10:27,218 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 01:10:27,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53775/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 01:10:27,568 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 01:10:28.187270 17928 mini_ranger_kms.cc:310] Created kms service
I20260812 01:10:28.450414 17928 mini_ranger_kms.cc:326] Created kudu user
I20260812 01:10:28.503666 17928 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 01:10:28 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497024, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:10:29.055990 17928 mini_ranger_kms.cc:384] Added ranger policy
I20260812 01:10:29.056126 17928 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms:/tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 01:10:29.056150 17928 mini_ranger_kms.cc:240] Using host: 127.17.130.20
I20260812 01:10:29.061702 17928 mini_ranger_kms.cc:276] Ranger KMS PID: 3412
I20260812 01:10:29.061857 17928 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.130.20:45103
Aug 12, 2026 1:10:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:10:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:10:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 1:10:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 1:10:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 1:10:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 01:10:31 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497031, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.130.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:10:32.142131 17928 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.775s user 0.009s sys 0.387s
I20260812 01:10:32.142263 17928 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 01:10:32.142364 17928 mini_ranger_kms.cc:401] 127.17.130.20:45103/kms/v1/keys
Aug 12 01:10:32 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497031, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.130.20@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
Aug 12 01:10:32 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497031, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.130.20@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
Aug 12 01:10:35 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:10:35.401028 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:41195
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:41195
--ranger_config_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:45103/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:10:35.504523 3513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:10:35.504868 3513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:10:35.504901 3513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:10:35.507274 3513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:10:35.507383 3513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:10:35.507400 3513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:10:35.507416 3513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:10:35.507428 3513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:10:35.509864 3513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.130.20:45103/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:41195
--ranger_config_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:41195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:10:35.510195 3513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:10:35.510491 3513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:10:35.516630 3523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:10:35.516620 3524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:10:35.516616 3529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:10:35.516765 3513 server_base.cc:1034] running on GCE node
I20260812 01:10:35.517340 3513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:10:35.517724 3513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:10:35.518994 3513 hybrid_clock.cc:648] HybridClock initialized: now 1786497035518942 us; error 65 us; skew 500 ppm
Aug 12 01:10:35 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497035, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:10:35.522504 3513 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:10:35.523042 3513 webserver.cc:460] Webserver started at http://127.17.130.62:45863/ using document root <none> and password file <none>
I20260812 01:10:35.523351 3513 fs_manager.cc:359] Metadata directory not provided
I20260812 01:10:35.523399 3513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:10:35.523556 3513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:10:35 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497035, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:10:35.596484 3513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "3da1090607b944318f31eb45077a3a5e"
format_stamp: "Formatted at 2026-08-12 01:10:35 on dist-test-slave-tfpf"
server_key: "86b1009c9973af92d3279e721ba4a6d4"
server_key_iv: "cbc48e1863a1e968da08df4569f76b7c"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:35.597527 3513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "3da1090607b944318f31eb45077a3a5e"
format_stamp: "Formatted at 2026-08-12 01:10:35 on dist-test-slave-tfpf"
server_key: "86b1009c9973af92d3279e721ba4a6d4"
server_key_iv: "cbc48e1863a1e968da08df4569f76b7c"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:35.600200 3513 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260812 01:10:35.622231 3541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:35.622470 3513 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:10:35.622605 3513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "3da1090607b944318f31eb45077a3a5e"
format_stamp: "Formatted at 2026-08-12 01:10:35 on dist-test-slave-tfpf"
server_key: "86b1009c9973af92d3279e721ba4a6d4"
server_key_iv: "cbc48e1863a1e968da08df4569f76b7c"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:35.622923 3513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:10:35.638108 3513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:10:35.640595 3513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:10:35.640812 3513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:10:35.646657 3513 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:41195
I20260812 01:10:35.646682 3633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:41195 every 8 connection(s)
I20260812 01:10:35.647223 3513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:10:35.648123 3639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:10:35.650812 3639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e: Bootstrap starting.
I20260812 01:10:35.651397 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 3513
I20260812 01:10:35.651564 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:10:35.651855 17928 external_mini_cluster.cc:1428] Setting key ac9b2ab6b35985b8f90db458318e8cfe
I20260812 01:10:35.652254 3639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e: Neither blocks nor log segments found. Creating new log.
I20260812 01:10:35.652760 3639 log.cc:824] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:10:35.654563 3639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e: No bootstrap required, opened a new log
I20260812 01:10:35.657531 3639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da1090607b944318f31eb45077a3a5e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41195 } }
I20260812 01:10:35.657752 3639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:10:35.657827 3639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da1090607b944318f31eb45077a3a5e, State: Initialized, Role: FOLLOWER
I20260812 01:10:35.657985 3639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da1090607b944318f31eb45077a3a5e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41195 } }
I20260812 01:10:35.658082 3639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:10:35.658123 3639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:10:35.658190 3639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:10:35.659229 3639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da1090607b944318f31eb45077a3a5e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41195 } }
I20260812 01:10:35.659363 3639 leader_election.cc:302] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3da1090607b944318f31eb45077a3a5e; no voters:
I20260812 01:10:35.659637 3639 leader_election.cc:288] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:10:35.659703 3648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:10:35.659978 3648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [term 1 LEADER]: Becoming Leader. State: Replica: 3da1090607b944318f31eb45077a3a5e, State: Running, Role: LEADER
I20260812 01:10:35.660147 3639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:10:35.660152 3648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da1090607b944318f31eb45077a3a5e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41195 } }
Aug 12 01:10:35 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:10:35.661661 3651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3da1090607b944318f31eb45077a3a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da1090607b944318f31eb45077a3a5e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41195 } } }
I20260812 01:10:35.661746 3651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [sys.catalog]: This master's current role is: LEADER
I20260812 01:10:35.661890 3649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3da1090607b944318f31eb45077a3a5e. Latest consensus state: current_term: 1 leader_uuid: "3da1090607b944318f31eb45077a3a5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da1090607b944318f31eb45077a3a5e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 41195 } } }
I20260812 01:10:35.661988 3656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:10:35.662011 3649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e [sys.catalog]: This master's current role is: LEADER
I20260812 01:10:35.662648 3656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:10:35.664315 3639 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 01:10:35.664484 3656 catalog_manager.cc:1347] Generated new cluster ID: ec20eaaf3423486e941c5676f990a108
I20260812 01:10:35.664538 3656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:10:35.669881 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:35.652739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.17.130.62:41195)
0812 01:10:35.653037 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:35.653051 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:35.653130 (+ 79us) server_negotiation.cc:407] Connection header received
0812 01:10:35.653754 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:35.653765 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:35.653834 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:35.654987 (+ 1153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:10:35.655118 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:35.656589 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:35.657819 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:35.658462 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:35.658588 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:35.661997 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:10:35.662002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:10:35.662002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:10:35.662018 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:10:35.664143 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:35.664592 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:35.664595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:35.664596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:35.664731 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:35.664930 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:35.664931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:35.664931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:35.667432 (+ 2501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:10:35.668295 (+ 863us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:35.668980 (+ 685us) server_negotiation.cc:299] Negotiation successful
0812 01:10:35.669524 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":144,"threads_started":1}
I20260812 01:10:35.681877 3656 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:10:35.682636 3656 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:10:35.704293 3656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e: Generated new TSK 0
I20260812 01:10:35.704631 3656 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:10:36.518822 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:45103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41195
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:10:36.657523 3714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:10:36.657914 3714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:10:36.657975 3714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:10:36.660192 3714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:10:36.660285 3714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:10:36.660344 3714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:10:36.663367 3714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.130.20:45103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:10:36.663755 3714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:10:36.664060 3714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:10:36.670485 3724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:10:36.670542 3729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:10:36.670715 3714 server_base.cc:1034] running on GCE node
W20260812 01:10:36.670485 3723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:10:36.671105 3714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:10:36.671383 3714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:10:36.672598 3714 hybrid_clock.cc:648] HybridClock initialized: now 1786497036672563 us; error 58 us; skew 500 ppm
Aug 12 01:10:36 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:10:36.675972 3714 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:10:36.676591 3714 webserver.cc:460] Webserver started at http://127.17.130.1:37255/ using document root <none> and password file <none>
I20260812 01:10:36.676900 3714 fs_manager.cc:359] Metadata directory not provided
I20260812 01:10:36.676983 3714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:10:36.677187 3714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:10:36 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:10:36.693392 3714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "f1882779cf4a40f7af593c1bd1bffae7"
format_stamp: "Formatted at 2026-08-12 01:10:36 on dist-test-slave-tfpf"
server_key: "df1c55ef0104e11893b429f1d015d712"
server_key_iv: "2cd3206908d1cf57b44a2da56c028f61"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:36.693871 3714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "f1882779cf4a40f7af593c1bd1bffae7"
format_stamp: "Formatted at 2026-08-12 01:10:36 on dist-test-slave-tfpf"
server_key: "df1c55ef0104e11893b429f1d015d712"
server_key_iv: "2cd3206908d1cf57b44a2da56c028f61"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:36.695751 3714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:10:36.712340 3740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:36.712546 3714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:10:36.712656 3714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "f1882779cf4a40f7af593c1bd1bffae7"
format_stamp: "Formatted at 2026-08-12 01:10:36 on dist-test-slave-tfpf"
server_key: "df1c55ef0104e11893b429f1d015d712"
server_key_iv: "2cd3206908d1cf57b44a2da56c028f61"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:36.712774 3714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:10:36.734510 3714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:10:36.737274 3714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:10:36.737465 3714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:10:36.737747 3714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:10:36.738193 3714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:10:36.738245 3714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:36.738276 3714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:10:36.738294 3714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:36.748687 3714 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:39143
I20260812 01:10:36.748839 3931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:39143 every 8 connection(s)
I20260812 01:10:36.749642 3714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:10:36 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:10:36.759254 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 3714
I20260812 01:10:36.759398 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:10:36.759660 17928 external_mini_cluster.cc:1428] Setting key f5367fc52b2ecb32b99e03dbfa3ffd38
I20260812 01:10:36.762997 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:36.751063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:38851 (local address 127.17.130.62:41195)
0812 01:10:36.751212 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:36.751216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:36.751474 (+ 258us) server_negotiation.cc:407] Connection header received
0812 01:10:36.752341 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:36.752343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:36.752360 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:36.752420 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:10:36.753110 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:36.754006 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:36.755187 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:36.755526 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:36.755623 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:36.758187 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:10:36.758188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:10:36.758189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:10:36.758199 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:10:36.760931 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:36.761554 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:36.761556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:36.761556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:36.761640 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:36.762020 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:36.762021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:36.762022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:36.762223 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:10:36.762320 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:36.762733 (+ 413us) server_negotiation.cc:299] Negotiation successful
0812 01:10:36.762849 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260812 01:10:36.763195 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:36.750990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41195 (local address 127.17.130.1:38851)
0812 01:10:36.751308 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:36.751367 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:36.752176 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:36.752517 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:36.752531 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:36.752751 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:10:36.752955 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:36.752959 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:36.754144 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:36.754145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:36.755045 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:36.755048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:36.755655 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:36.755656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:36.755880 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:36.756541 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:10:36.756550 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:10:36.758052 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:10:36.761037 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:10:36.761050 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:10:36.761054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:10:36.761458 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:10:36.761717 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:10:36.761718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:10:36.761718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:10:36.761886 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:10:36.762324 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:10:36.762332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:10:36.762531 (+ 199us) client_negotiation.cc:769] Sending connection context
0812 01:10:36.762671 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 01:10:36.762873 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":111,"threads_started":1}
I20260812 01:10:36.764163 3933 heartbeater.cc:347] Connected to a master server at 127.17.130.62:41195
I20260812 01:10:36.764288 3933 heartbeater.cc:464] Registering TS with master...
I20260812 01:10:36.764521 3933 heartbeater.cc:511] Master 127.17.130.62:41195 requested a full tablet report, sending...
I20260812 01:10:36.765194 3570 ts_manager.cc:194] Registered new tserver with Master: f1882779cf4a40f7af593c1bd1bffae7 (127.17.130.1:39143)
I20260812 01:10:36.765846 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:38851
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:10:36.797569 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:45103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41195
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:10:36 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 01:10:36.936234 3939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:10:36.936571 3939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:10:36.936611 3939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:10:36.938586 3939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:10:36.938661 3939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:10:36.938706 3939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:10:36.941203 3939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.130.20:45103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:10:36.941496 3939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:10:36.941778 3939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:10:36.950502 3947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:10:36.950665 3939 server_base.cc:1034] running on GCE node
W20260812 01:10:36.950513 3948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:10:36.953260 3954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:10:36.953619 3939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:10:36.953940 3939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:10:36.955114 3939 hybrid_clock.cc:648] HybridClock initialized: now 1786497036955080 us; error 56 us; skew 500 ppm
Aug 12 01:10:36 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:10:36.958873 3939 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:10:36.959491 3939 webserver.cc:460] Webserver started at http://127.17.130.2:37079/ using document root <none> and password file <none>
I20260812 01:10:36.959793 3939 fs_manager.cc:359] Metadata directory not provided
I20260812 01:10:36.959860 3939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:10:36.960011 3939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:10:36 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:10:36.979779 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "cc8f442eca3d447fa331811fe2300ad3"
format_stamp: "Formatted at 2026-08-12 01:10:36 on dist-test-slave-tfpf"
server_key: "2bbf6e9c943877142b91582df14f3c13"
server_key_iv: "fbe0fa6a853265ebe068a5ae9bdabed0"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:36.980401 3939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "cc8f442eca3d447fa331811fe2300ad3"
format_stamp: "Formatted at 2026-08-12 01:10:36 on dist-test-slave-tfpf"
server_key: "2bbf6e9c943877142b91582df14f3c13"
server_key_iv: "fbe0fa6a853265ebe068a5ae9bdabed0"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:36.986564 3939 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260812 01:10:37.012104 3965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:37.012339 3939 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260812 01:10:37.012451 3939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "cc8f442eca3d447fa331811fe2300ad3"
format_stamp: "Formatted at 2026-08-12 01:10:36 on dist-test-slave-tfpf"
server_key: "2bbf6e9c943877142b91582df14f3c13"
server_key_iv: "fbe0fa6a853265ebe068a5ae9bdabed0"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:37.012576 3939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:10:37.025257 3939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:10:37.028452 3939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:10:37.028637 3939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:10:37.028983 3939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:10:37.029448 3939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:10:37.029497 3939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:37.029533 3939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:10:37.029553 3939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:37.038316 3939 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:34319
I20260812 01:10:37.039104 3939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:10:37.039798 4171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:34319 every 8 connection(s)
Aug 12 01:10:37 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:10:37.048851 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 3939
I20260812 01:10:37.048997 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:10:37.049265 17928 external_mini_cluster.cc:1428] Setting key 019544b6be125d3e01bb7207db651639
I20260812 01:10:37.053454 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.040575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:33767 (local address 127.17.130.62:41195)
0812 01:10:37.040652 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.040655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.040947 (+ 292us) server_negotiation.cc:407] Connection header received
0812 01:10:37.041730 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.041731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.041749 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.041810 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:10:37.042469 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.042998 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.044641 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.044917 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.045038 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.048885 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:10:37.048887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:10:37.048887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:10:37.048898 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:10:37.051261 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:37.051966 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:37.051967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:37.051968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:37.052043 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:37.052386 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:37.052387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:37.052387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:37.052565 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:10:37.052622 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.053182 (+ 560us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.053314 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:10:37.054137 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.040588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41195 (local address 127.17.130.2:33767)
0812 01:10:37.040779 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.040823 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.041581 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.041895 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.041900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.042123 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:10:37.042338 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.042343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.043113 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.043114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.044485 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.044489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.045079 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.045081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.045229 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.046466 (+ 1237us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:10:37.046476 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:10:37.048697 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:10:37.051409 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:10:37.051424 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:10:37.051427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:10:37.051835 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:10:37.052133 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:10:37.052134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:10:37.052134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:10:37.052299 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:10:37.052701 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:10:37.052710 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:10:37.052940 (+ 230us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.053596 (+ 656us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.053809 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":109,"threads_started":1}
I20260812 01:10:37.054838 4173 heartbeater.cc:347] Connected to a master server at 127.17.130.62:41195
I20260812 01:10:37.054968 4173 heartbeater.cc:464] Registering TS with master...
I20260812 01:10:37.055210 4173 heartbeater.cc:511] Master 127.17.130.62:41195 requested a full tablet report, sending...
I20260812 01:10:37.055693 3570 ts_manager.cc:194] Registered new tserver with Master: cc8f442eca3d447fa331811fe2300ad3 (127.17.130.2:34319)
I20260812 01:10:37.056231 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:33767
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:10:37.093427 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:45103/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41195
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:10:37.235296 4179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:10:37.235716 4179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:10:37.235811 4179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:10:37.237875 4179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:10:37.238214 4179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:10:37.238337 4179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:10:37.242396 4179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.130.20:45103/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:41195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:10:37.243083 4179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:10:37.243507 4179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:10:37.252357 4188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:10:37.252269 4185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:10:37.252269 4186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:10:37.254027 4179 server_base.cc:1034] running on GCE node
I20260812 01:10:37.254328 4179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:10:37.254689 4179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:10:37.258016 4179 hybrid_clock.cc:648] HybridClock initialized: now 1786497037257990 us; error 52 us; skew 500 ppm
Aug 12 01:10:37 dist-test-slave-tfpf krb5kdc[2774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497037, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:10:37.262217 4179 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:10:37.263026 4179 webserver.cc:460] Webserver started at http://127.17.130.3:46131/ using document root <none> and password file <none>
I20260812 01:10:37.263417 4179 fs_manager.cc:359] Metadata directory not provided
I20260812 01:10:37.263556 4179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:10:37.263773 4179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:10:37 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497037, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:10:37.282948 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "f7e5fb5a8a73444cb16ebb015935ae57"
format_stamp: "Formatted at 2026-08-12 01:10:37 on dist-test-slave-tfpf"
server_key: "cd012e5433257656fa2ce10d2d2d00f1"
server_key_iv: "a583ec61858dff9069d991af7468cee0"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:37.283633 4179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "f7e5fb5a8a73444cb16ebb015935ae57"
format_stamp: "Formatted at 2026-08-12 01:10:37 on dist-test-slave-tfpf"
server_key: "cd012e5433257656fa2ce10d2d2d00f1"
server_key_iv: "a583ec61858dff9069d991af7468cee0"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:37.287690 4179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260812 01:10:37.313596 4200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:37.313835 4179 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260812 01:10:37.313951 4179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "f7e5fb5a8a73444cb16ebb015935ae57"
format_stamp: "Formatted at 2026-08-12 01:10:37 on dist-test-slave-tfpf"
server_key: "cd012e5433257656fa2ce10d2d2d00f1"
server_key_iv: "a583ec61858dff9069d991af7468cee0"
server_key_version: "kuduclusterkey@0"
I20260812 01:10:37.314087 4179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:10:37.332105 4179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:10:37.335951 4179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:10:37.336203 4179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:10:37.336520 4179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:10:37.337149 4179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:10:37.337337 4179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:37.337461 4179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:10:37.337487 4179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:10:37.346503 4179 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:38417
I20260812 01:10:37.347036 4179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:10:37.350173 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:38417 every 8 connection(s)
I20260812 01:10:37.353475 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 4179
I20260812 01:10:37.353618 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:10:37.353818 17928 external_mini_cluster.cc:1428] Setting key e72b047e190f5c7cd006cb2707072adb
Aug 12 01:10:37 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497037, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:10:37.367726 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.355477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:52401 (local address 127.17.130.62:41195)
0812 01:10:37.355528 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.355531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.356312 (+ 781us) server_negotiation.cc:407] Connection header received
0812 01:10:37.357134 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.357135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.357148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.357184 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:10:37.358159 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.358925 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.360588 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.360884 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.360935 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.363954 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:10:37.363956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:10:37.363957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:10:37.363966 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:10:37.365884 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:37.366460 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:37.366462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:37.366462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:37.366532 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:37.366878 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:37.366879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:37.366879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:37.367038 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:10:37.367101 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.367521 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.367620 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:10:37.368655 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.355655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:41195 (local address 127.17.130.3:52401)
0812 01:10:37.356180 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.356225 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.356963 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.357392 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.357396 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.357724 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:10:37.358039 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.358045 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.359055 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.359056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.360473 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.360477 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.361038 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.361039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.361276 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.362024 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:10:37.362036 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:10:37.363816 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:10:37.365991 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:10:37.365999 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:10:37.366002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:10:37.366355 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:10:37.366617 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:10:37.366618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:10:37.366618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:10:37.366773 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:10:37.367159 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:10:37.367167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:10:37.367353 (+ 186us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.368175 (+ 822us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.368378 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":197,"threads_started":1}
I20260812 01:10:37.369483 4393 heartbeater.cc:347] Connected to a master server at 127.17.130.62:41195
I20260812 01:10:37.369594 4393 heartbeater.cc:464] Registering TS with master...
I20260812 01:10:37.369824 4393 heartbeater.cc:511] Master 127.17.130.62:41195 requested a full tablet report, sending...
I20260812 01:10:37.370311 3570 ts_manager.cc:194] Registered new tserver with Master: f7e5fb5a8a73444cb16ebb015935ae57 (127.17.130.3:38417)
I20260812 01:10:37.370803 3570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:52401
I20260812 01:10:37.376588 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:10:37.388230 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.378862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58598 (local address 127.17.130.62:41195)
0812 01:10:37.378920 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.378922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.379240 (+ 318us) server_negotiation.cc:407] Connection header received
0812 01:10:37.379361 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.379362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.379376 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.379410 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:10:37.380416 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.381281 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.382409 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.382707 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.382762 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.384591 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:10:37.384593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:10:37.384593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:10:37.384602 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:10:37.386507 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:37.386967 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:37.386968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:37.386968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:37.387035 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:10:37.387271 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:10:37.387272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:10:37.387272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:10:37.387421 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:10:37.387532 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.388016 (+ 484us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.388125 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:10:37.390981 3570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:10:37.393605 3570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:10:37.415901 4418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.397978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:38417 (local address 127.0.0.1:47424)
0812 01:10:37.401671 (+ 3693us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.401682 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.401738 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.402042 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.402043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.402045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:10:37.402119 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.402121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.409257 (+ 7136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.409258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.410814 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.410817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.415437 (+ 4620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.415438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.415522 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.415528 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.415588 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.415789 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3672,"thread_start_us":77,"threads_started":1}
I20260812 01:10:37.416451 4419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.397475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39143 (local address 127.0.0.1:58600)
0812 01:10:37.399293 (+ 1818us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.399347 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.399427 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.400057 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.400061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.400063 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:10:37.400162 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.400164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.407234 (+ 7070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.407235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.408468 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.408473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.413607 (+ 5134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.413608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.413742 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.413762 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.416315 (+ 2553us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.416380 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1780,"thread_start_us":97,"threads_started":1}
I20260812 01:10:37.416952 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.397781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:34319 (local address 127.0.0.1:41170)
0812 01:10:37.401260 (+ 3479us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.401272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.401330 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.403570 (+ 2240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.403571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.403573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:10:37.403622 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.403624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.404960 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.404961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.412336 (+ 7375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.412340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.416592 (+ 4252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.416593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.416702 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.416707 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.416776 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.416826 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3456,"thread_start_us":78,"threads_started":1}
I20260812 01:10:37.418237 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.398314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.17.130.2:34319)
0812 01:10:37.399600 (+ 1286us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.399607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.401395 (+ 1788us) server_negotiation.cc:407] Connection header received
0812 01:10:37.401448 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.401451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.401469 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.401546 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:10:37.403715 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.404843 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.412500 (+ 7657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.413383 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.413470 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.413556 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.418073 (+ 4517us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.418143 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1242,"thread_start_us":101,"threads_started":1}
I20260812 01:10:37.418668 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.398698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.17.130.1:39143)
0812 01:10:37.399677 (+ 979us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.399680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.399692 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:10:37.399733 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.399736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.399754 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.399844 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:10:37.400262 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.407045 (+ 6783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.410960 (+ 3915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.411854 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.411941 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.412015 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.418502 (+ 6487us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.418581 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":78,"threads_started":1}
I20260812 01:10:37.419893 4036 tablet_service.cc:1467] Processing CreateTablet for tablet 6aa6ccdff1ff44aa96109e30fb1e261f (DEFAULT_TABLE table=test-table [id=f4488c5660f74e59bd222c1a8c44f628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:10:37.420409 4036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aa6ccdff1ff44aa96109e30fb1e261f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:10:37.429190 3786 tablet_service.cc:1467] Processing CreateTablet for tablet 6aa6ccdff1ff44aa96109e30fb1e261f (DEFAULT_TABLE table=test-table [id=f4488c5660f74e59bd222c1a8c44f628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:10:37.429724 3786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aa6ccdff1ff44aa96109e30fb1e261f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:10:37.435092 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.398871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47424 (local address 127.17.130.3:38417)
0812 01:10:37.400006 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.400011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.401801 (+ 1790us) server_negotiation.cc:407] Connection header received
0812 01:10:37.401851 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.401853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.401872 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.401946 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:10:37.402213 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.409127 (+ 6914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.414335 (+ 5208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.415214 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.415303 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.415389 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.434887 (+ 19498us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.434972 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":105,"threads_started":1}
I20260812 01:10:37.436224 4261 tablet_service.cc:1467] Processing CreateTablet for tablet 6aa6ccdff1ff44aa96109e30fb1e261f (DEFAULT_TABLE table=test-table [id=f4488c5660f74e59bd222c1a8c44f628]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:10:37.436743 4261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aa6ccdff1ff44aa96109e30fb1e261f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:10:37.442512 4426 tablet_bootstrap.cc:492] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: Bootstrap starting.
I20260812 01:10:37.443710 4426 tablet_bootstrap.cc:654] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: Neither blocks nor log segments found. Creating new log.
I20260812 01:10:37.444077 4425 tablet_bootstrap.cc:492] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: Bootstrap starting.
I20260812 01:10:37.445449 4427 tablet_bootstrap.cc:492] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Bootstrap starting.
I20260812 01:10:37.446519 4426 log.cc:824] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:10:37.449723 4427 tablet_bootstrap.cc:654] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Neither blocks nor log segments found. Creating new log.
I20260812 01:10:37.450845 4425 tablet_bootstrap.cc:654] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: Neither blocks nor log segments found. Creating new log.
I20260812 01:10:37.451846 4425 log.cc:824] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: Log is configured to *not* fsync() on all Append() calls
I20260812 01:10:37.452666 4427 log.cc:824] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Log is configured to *not* fsync() on all Append() calls
I20260812 01:10:37.454511 4426 tablet_bootstrap.cc:492] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: No bootstrap required, opened a new log
I20260812 01:10:37.454720 4426 ts_tablet_manager.cc:1397] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260812 01:10:37.458001 4426 raft_consensus.cc:348] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.458482 4426 raft_consensus.cc:374] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:10:37.458616 4426 raft_consensus.cc:729] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1882779cf4a40f7af593c1bd1bffae7, State: Initialized, Role: FOLLOWER
I20260812 01:10:37.458881 4426 consensus_queue.cc:260] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.460203 4426 ts_tablet_manager.cc:1428] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260812 01:10:37.460410 3933 heartbeater.cc:503] Master 127.17.130.62:41195 was elected leader, sending a full tablet report...
I20260812 01:10:37.463268 4425 tablet_bootstrap.cc:492] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: No bootstrap required, opened a new log
I20260812 01:10:37.463486 4425 ts_tablet_manager.cc:1397] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.001s
I20260812 01:10:37.467433 4425 raft_consensus.cc:348] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.468055 4425 raft_consensus.cc:374] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:10:37.469022 4427 tablet_bootstrap.cc:492] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: No bootstrap required, opened a new log
I20260812 01:10:37.469314 4427 ts_tablet_manager.cc:1397] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260812 01:10:37.472733 4427 raft_consensus.cc:348] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.473239 4427 raft_consensus.cc:374] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:10:37.473356 4427 raft_consensus.cc:729] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e5fb5a8a73444cb16ebb015935ae57, State: Initialized, Role: FOLLOWER
I20260812 01:10:37.473639 4427 consensus_queue.cc:260] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.473862 4425 raft_consensus.cc:729] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc8f442eca3d447fa331811fe2300ad3, State: Initialized, Role: FOLLOWER
I20260812 01:10:37.474315 4427 ts_tablet_manager.cc:1428] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260812 01:10:37.474578 4425 consensus_queue.cc:260] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.475500 4393 heartbeater.cc:503] Master 127.17.130.62:41195 was elected leader, sending a full tablet report...
I20260812 01:10:37.475768 4425 ts_tablet_manager.cc:1428] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: Time spent starting tablet: real 0.012s user 0.003s sys 0.002s
I20260812 01:10:37.476437 4173 heartbeater.cc:503] Master 127.17.130.62:41195 was elected leader, sending a full tablet report...
W20260812 01:10:37.500798 3934 tablet.cc:2369] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:10:37.547519 4175 tablet.cc:2369] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 01:10:37 dist-test-slave-tfpf krb5kdc[2774](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497036, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
W20260812 01:10:37.606433 4394 tablet.cc:2369] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:10:37.723660 4437 raft_consensus.cc:482] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:10:37.723861 4437 raft_consensus.cc:504] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.724298 4437 leader_election.cc:288] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1882779cf4a40f7af593c1bd1bffae7 (127.17.130.1:39143), cc8f442eca3d447fa331811fe2300ad3 (127.17.130.2:34319)
I20260812 01:10:37.729841 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.724568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39143 (local address 127.17.130.3:57941)
0812 01:10:37.724620 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.724636 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.724691 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.724925 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.724926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.724928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:10:37.725007 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.725010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.725961 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.725962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.728003 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.728013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.729478 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.729480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.729605 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.729612 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.729697 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.729753 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 01:10:37.730494 4421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.724737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:57941 (local address 127.17.130.1:39143)
0812 01:10:37.724787 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.724789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.724797 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:10:37.724831 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.724832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.724845 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.724912 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:10:37.725116 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.725846 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.728224 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.728999 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.729125 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.729391 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.730346 (+ 955us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.730425 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:10:37.730801 3853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa6ccdff1ff44aa96109e30fb1e261f" candidate_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1882779cf4a40f7af593c1bd1bffae7" is_pre_election: true
I20260812 01:10:37.730973 3853 raft_consensus.cc:2454] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e5fb5a8a73444cb16ebb015935ae57 in term 0.
I20260812 01:10:37.731276 4212 leader_election.cc:302] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1882779cf4a40f7af593c1bd1bffae7, f7e5fb5a8a73444cb16ebb015935ae57; no voters:
I20260812 01:10:37.731422 4437 raft_consensus.cc:2781] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:10:37.731464 4437 raft_consensus.cc:482] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:10:37.731485 4437 raft_consensus.cc:3037] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:10:37.732458 4437 raft_consensus.cc:504] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.732617 4437 leader_election.cc:288] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [CANDIDATE]: Term 1 election: Requested vote from peers f1882779cf4a40f7af593c1bd1bffae7 (127.17.130.1:39143), cc8f442eca3d447fa331811fe2300ad3 (127.17.130.2:34319)
I20260812 01:10:37.733210 3853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa6ccdff1ff44aa96109e30fb1e261f" candidate_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1882779cf4a40f7af593c1bd1bffae7"
I20260812 01:10:37.733299 3853 raft_consensus.cc:3037] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:10:37.734275 3853 raft_consensus.cc:2454] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e5fb5a8a73444cb16ebb015935ae57 in term 1.
I20260812 01:10:37.734570 4212 leader_election.cc:302] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f1882779cf4a40f7af593c1bd1bffae7, f7e5fb5a8a73444cb16ebb015935ae57; no voters:
I20260812 01:10:37.734709 4437 raft_consensus.cc:2781] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:10:37.734889 4437 raft_consensus.cc:686] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 1 LEADER]: Becoming Leader. State: Replica: f7e5fb5a8a73444cb16ebb015935ae57, State: Running, Role: LEADER
I20260812 01:10:37.734993 4437 consensus_queue.cc:237] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } }
I20260812 01:10:37.735968 3570 catalog_manager.cc:5495] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7e5fb5a8a73444cb16ebb015935ae57 (127.17.130.3). New cstate: current_term: 1 leader_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 38417 } health_report { overall_health: HEALTHY } } }
I20260812 01:10:37.740710 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.725105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:34319 (local address 127.17.130.3:49695)
0812 01:10:37.729954 (+ 4849us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.729967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:10:37.730021 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:10:37.737439 (+ 7418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:10:37.737441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:10:37.737444 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:10:37.737499 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.737501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.738467 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.738468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.739565 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:10:37.739568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.740466 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.740467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:10:37.740531 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.740535 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:10:37.740578 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 01:10:37.740623 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4836,"thread_start_us":172,"threads_started":1}
I20260812 01:10:37.741130 4420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.725392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:49695 (local address 127.17.130.2:34319)
0812 01:10:37.725436 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.725438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.730181 (+ 4743us) server_negotiation.cc:407] Connection header received
0812 01:10:37.730219 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.730220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.730237 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.730284 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:10:37.737605 (+ 7321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.738333 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.739696 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.740334 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.740391 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.740401 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.740985 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.741034 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:10:37.741725 4071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa6ccdff1ff44aa96109e30fb1e261f" candidate_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc8f442eca3d447fa331811fe2300ad3" is_pre_election: true
I20260812 01:10:37.741914 4071 raft_consensus.cc:2454] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7e5fb5a8a73444cb16ebb015935ae57 in term 0.
I20260812 01:10:37.742200 4073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aa6ccdff1ff44aa96109e30fb1e261f" candidate_uuid: "f7e5fb5a8a73444cb16ebb015935ae57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc8f442eca3d447fa331811fe2300ad3"
I20260812 01:10:37.742291 4073 raft_consensus.cc:3037] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:10:37.743359 4073 raft_consensus.cc:2454] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7e5fb5a8a73444cb16ebb015935ae57 in term 1.
I20260812 01:10:37.768182 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.764402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.17.130.3:38417)
0812 01:10:37.764450 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:10:37.764452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:10:37.764466 (+ 14us) server_negotiation.cc:407] Connection header received
0812 01:10:37.764539 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:10:37.764540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:10:37.764569 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:10:37.764652 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:10:37.764928 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.765709 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.766839 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:10:37.767184 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:10:37.767253 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:10:37.767489 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:10:37.767643 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:10:37.767732 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 01:10:37.768023 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 01:10:37.768053 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:10:37.778599 4073 raft_consensus.cc:1261] T 6aa6ccdff1ff44aa96109e30fb1e261f P cc8f442eca3d447fa331811fe2300ad3 [term 1 FOLLOWER]: Refusing update from remote peer f7e5fb5a8a73444cb16ebb015935ae57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:10:37.779023 3853 raft_consensus.cc:1261] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 1 FOLLOWER]: Refusing update from remote peer f7e5fb5a8a73444cb16ebb015935ae57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:10:37.779420 4437 consensus_queue.cc:1035] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1882779cf4a40f7af593c1bd1bffae7" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 39143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:10:37.779908 4437 consensus_queue.cc:1035] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc8f442eca3d447fa331811fe2300ad3" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:10:37.794107 4450 mvcc.cc:205] Tried to move back new op lower bound from 7317491866736803840 to 7317491866563145728. Current Snapshot: MvccSnapshot[applied={T|T < 7317491866736803840}]
I20260812 01:10:37.794590 4451 mvcc.cc:205] Tried to move back new op lower bound from 7317491866736803840 to 7317491866563145728. Current Snapshot: MvccSnapshot[applied={T|T < 7317491866736803840}]
I20260812 01:10:37.809314 3570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:10:37.809454 3570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:10:37.811038 3570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3da1090607b944318f31eb45077a3a5e: Sending DeleteTablet for 3 replicas of tablet 6aa6ccdff1ff44aa96109e30fb1e261f
I20260812 01:10:37.811617 3786 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aa6ccdff1ff44aa96109e30fb1e261f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:10:37 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:58600
I20260812 01:10:37.811636 4036 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aa6ccdff1ff44aa96109e30fb1e261f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:10:37 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:41170
I20260812 01:10:37.812273 4261 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aa6ccdff1ff44aa96109e30fb1e261f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:10:37 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:47424
I20260812 01:10:37.812752 4474 tablet_replica.cc:330] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7: stopping tablet replica
I20260812 01:10:37.812875 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 3714
I20260812 01:10:37.812933 4474 raft_consensus.cc:2229] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:10:37.813094 4474 raft_consensus.cc:2258] T 6aa6ccdff1ff44aa96109e30fb1e261f P f1882779cf4a40f7af593c1bd1bffae7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 01:10:37.827696 3544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:39143 (error 108)
I20260812 01:10:37.828178 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 3939
I20260812 01:10:37.849982 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.849822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39143 (local address 127.0.0.1:58606)
0812 01:10:37.849885 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.849928 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 01:10:37.850313 3544 catalog_manager.cc:4584] TS f1882779cf4a40f7af593c1bd1bffae7 (127.17.130.1:39143): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6aa6ccdff1ff44aa96109e30fb1e261f: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39143: connect: Connection refused (error 111)
I20260812 01:10:37.850617 4476 tablet_replica.cc:330] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: stopping tablet replica
I20260812 01:10:37.857208 4476 raft_consensus.cc:2229] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:10:37.857762 4476 raft_consensus.cc:2258] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:10:37.858919 4476 ts_tablet_manager.cc:1905] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:10:37.873270 4476 ts_tablet_manager.cc:1918] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:10:37.873528 4476 log.cc:1196] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/6aa6ccdff1ff44aa96109e30fb1e261f
I20260812 01:10:37.873942 4476 ts_tablet_manager.cc:1939] T 6aa6ccdff1ff44aa96109e30fb1e261f P f7e5fb5a8a73444cb16ebb015935ae57: Deleting consensus metadata
I20260812 01:10:37.874621 3543 catalog_manager.cc:4841] TS f7e5fb5a8a73444cb16ebb015935ae57 (127.17.130.3:38417): tablet 6aa6ccdff1ff44aa96109e30fb1e261f (table test-table [id=f4488c5660f74e59bd222c1a8c44f628]) successfully deleted
I20260812 01:10:37.875146 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.875020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:34319 (local address 127.0.0.1:41182)
0812 01:10:37.875086 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.875114 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.2:34319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 01:10:37.881479 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 4179
I20260812 01:10:37.903611 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.903200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39143 (local address 127.0.0.1:58610)
0812 01:10:37.903287 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.903335 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 01:10:37.904354 4417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:10:37.904223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:39143 (local address 127.0.0.1:58622)
0812 01:10:37.904285 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:10:37.904321 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:39143: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:10:37.911437 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 3513
I20260812 01:10:37.932989 17928 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 01:10:38.289705 17928 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 01:10:38.289896 17928 mini_ranger.cc:66] Stopping Ranger...
I20260812 01:10:38.648838 17928 mini_ranger.cc:68] Stopped Ranger
2026-08-12 01:10:38.649 UTC [2815] LOG: received smart shutdown request
2026-08-12 01:10:38.654 UTC [2815] LOG: background worker "logical replication launcher" (PID 2833) exited with exit code 1
2026-08-12 01:10:38.654 UTC [2823] LOG: shutting down
2026-08-12 01:10:38.855 UTC [2815] LOG: database system is shut down
2026-08-12T01:10:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54683 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:10:39 dist-test-slave-tfpf krb5kdc[4489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:10:39 dist-test-slave-tfpf krb5kdc[4489](info): set up 2 sockets
Aug 12 01:10:39 dist-test-slave-tfpf krb5kdc[4489](info): commencing operation
krb5kdc: starting...
W20260812 01:10:40.371515 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:10:40 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:10:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:10:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/rangeradmin_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/rangeradmin_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/rangerlookup_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/rangerlookup_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
I20260812 01:10:40.576963 17928 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskolrujf/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/postgres -l logfile start
2026-08-12 01:10:43.465 UTC [4530] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 01:10:43.465 UTC [4530] LOG: listening on IPv4 address "127.17.130.20", port 53031
2026-08-12 01:10:43.469 UTC [4530] LOG: listening on Unix socket "/tmp/.s.PGSQL.53031"
2026-08-12 01:10:43.512 UTC [4534] LOG: database system was shut down at 2026-08-12 01:10:41 UTC
2026-08-12 01:10:43.516 UTC [4530] LOG: database system is ready to accept connections
I20260812 01:10:44.729895 17928 mini_postgres.cc:96] Postgres bound to 53031
2026-08-12 01:10:44.739 UTC [4545] FATAL: database "slave" does not exist
127.17.130.20:53031 - accepting connections
I20260812 01:10:44.740546 17928 mini_ranger.cc:161] Starting Ranger...
I20260812 01:10:44.761128 17928 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 01:10:45.357867 17928 mini_ranger.cc:87] Created ranger Postgres database
I20260812 01:10:45.357998 17928 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-admin
2026-08-12 01:10:45,672 [I] DB FLAVOR :POSTGRES
2026-08-12 01:10:45,673 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:10:45,673 [I] Checking connection..
2026-08-12 01:10:45,673 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 01:10:45,972 [I] Checking connection passed.
2026-08-12 01:10:45,973 [I] --------- Verifying version history table ---------
2026-08-12 01:10:45,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:10:46,261 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:10:46,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:10:46,560 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 01:10:46,561 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 01:10:46,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 01:10:46.849 UTC [4659] WARNING: there is no transaction in progress
2026-08-12 01:10:46,856 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 01:10:46,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 01:10:47,129 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 01:10:47,130 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:10:47,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 01:10:47,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 01:10:47,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-tfpf.c.gcp-upstream.internal','N') ;"
2026-08-12 01:10:47,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:10:48,271 [I] Table x_portal_user does not exist in database ranger
2026-08-12 01:10:48,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:10:48,559 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 01:10:48,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:10:48,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:10:49,127 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 01:10:49,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 01:10:50.238 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.257 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.276 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.294 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.313 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.335 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.413 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.421 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.437 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.449 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.460 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.476 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.488 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.498 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.507 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:50.516 UTC [4838] WARNING: there is no transaction in progress
2026-08-12 01:10:51,333 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 01:10:51,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 01:10:51,609 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 01:10:51,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 01:10:51,894 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 01:10:51,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 01:10:52,174 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 01:10:52,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 01:10:52,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 01:10:52,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-tfpf.c.gcp-upstream.internal';"
2026-08-12 01:10:53,000 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 01:10:53,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 01:10:53,291 [I] DB_PATCHES have already been applied
I20260812 01:10:53.297694 17928 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-admin:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskolrujf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 01:10:53.297822 17928 mini_ranger.cc:193] Using host: 127.17.130.20
I20260812 01:10:53.303058 17928 mini_ranger.cc:227] Ranger admin URL: http://127.17.130.20:50995
Aug 12, 2026 1:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 1:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 1:10:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 1:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 1:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 01:10:56.325444 17928 mini_ranger.cc:160] Time spent starting Ranger: real 11.585s user 0.001s sys 0.015s
Aug 12, 2026 1:11:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 1:11:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 1:11:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 1:11:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 1:11:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 01:11:20.067845 17928 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.17.130.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.17.130.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/rangerkms_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.17.130.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/rangerkms_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.17.130.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.17.130.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.17.130.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/HTTP_127.17.130.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 01:11:20 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497080, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 01:11:20.174436 17928 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 01:11:20.188769 17928 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 01:11:21.222903 17928 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 01:11:21.223023 17928 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms
I20260812 01:11:21.223047 17928 mini_ranger_kms.cc:227] Using postgres at 127.17.130.20:53031
2026-08-12 01:11:21,421 [I] DB FLAVOR :POSTGRES
2026-08-12 01:11:21,421 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 01:11:21,421 [I] Checking connection
2026-08-12 01:11:21,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 01:11:21,715 [I] connection success
2026-08-12 01:11:21,716 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 01:11:21,716 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 01:11:21,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 01:11:22,029 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 01:11:22,029 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 01:11:22,029 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 01:11:22,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.17.130.20:53031/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 01:11:22,363 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 01:11:22.950333 17928 mini_ranger_kms.cc:310] Created kms service
I20260812 01:11:23.178397 17928 mini_ranger_kms.cc:326] Created kudu user
I20260812 01:11:23.243173 17928 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 01:11:23 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497080, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:23.728533 17928 mini_ranger_kms.cc:384] Added ranger policy
I20260812 01:11:23.728775 17928 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms:/tmp/dist-test-taskolrujf/build/release/bin/postgresql.jar:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskolrujf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 01:11:23.728801 17928 mini_ranger_kms.cc:240] Using host: 127.17.130.20
I20260812 01:11:23.734122 17928 mini_ranger_kms.cc:276] Ranger KMS PID: 5124
I20260812 01:11:23.734231 17928 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.17.130.20:46373
Aug 12, 2026 1:11:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 1:11:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 1:11:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 1:11:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 1:11:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 1:11:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 01:11:26 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497086, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.130.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:26.697491 17928 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.523s user 0.000s sys 0.181s
I20260812 01:11:26.697615 17928 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 01:11:26.697675 17928 mini_ranger_kms.cc:401] 127.17.130.20:46373/kms/v1/keys
Aug 12 01:11:26 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497086, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.130.20@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
Aug 12 01:11:27 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497086, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.17.130.20@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
Aug 12 01:11:29 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:11:29.567760 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44467
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44467
--ranger_config_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:29.658983 5198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:29.659226 5198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:29.659250 5198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:29.660725 5198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:29.660800 5198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:29.660820 5198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:29.660840 5198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:29.660853 5198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:29.662859 5198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
--ranger_kms_url=127.17.130.20:46373/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44467
--ranger_config_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:29.663179 5198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:29.663452 5198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:11:29.669395 5198 server_base.cc:1034] running on GCE node
W20260812 01:11:29.669364 5207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:29.669353 5212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:29.669368 5206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:29.670128 5198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:29.670444 5198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:29.671636 5198 hybrid_clock.cc:648] HybridClock initialized: now 1786497089671618 us; error 44 us; skew 500 ppm
Aug 12 01:11:29 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:29.679337 5198 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:29.679829 5198 webserver.cc:460] Webserver started at http://127.17.130.62:41717/ using document root <none> and password file <none>
I20260812 01:11:29.680074 5198 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:29.680119 5198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:29.680262 5198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:11:29 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497089, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:29.743474 5198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1"
format_stamp: "Formatted at 2026-08-12 01:11:29 on dist-test-slave-tfpf"
server_key: "3849ab10496ec6bf84097cfd2c24a7b4"
server_key_iv: "ac8e56cac4dc3dfeabed4f6e5853c9aa"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:29.744187 5198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1"
format_stamp: "Formatted at 2026-08-12 01:11:29 on dist-test-slave-tfpf"
server_key: "3849ab10496ec6bf84097cfd2c24a7b4"
server_key_iv: "ac8e56cac4dc3dfeabed4f6e5853c9aa"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:29.746609 5198 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260812 01:11:29.763437 5225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:29.763631 5198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:11:29.763734 5198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1"
format_stamp: "Formatted at 2026-08-12 01:11:29 on dist-test-slave-tfpf"
server_key: "3849ab10496ec6bf84097cfd2c24a7b4"
server_key_iv: "ac8e56cac4dc3dfeabed4f6e5853c9aa"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:29.763835 5198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:29.782306 5198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:29.785220 5198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:29.785408 5198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:29.790848 5198 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44467
I20260812 01:11:29.790923 5309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44467 every 8 connection(s)
I20260812 01:11:29.791421 5198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:29.792335 5311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:29.794848 5311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Bootstrap starting.
I20260812 01:11:29.795504 5311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:29.795826 5311 log.cc:824] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:29.796089 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 5198
I20260812 01:11:29.796202 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:11:29.796387 17928 external_mini_cluster.cc:1428] Setting key 1263813a6344ec95ae2356d7060e8d9e
I20260812 01:11:29.796703 5311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: No bootstrap required, opened a new log
I20260812 01:11:29.799758 5311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:29.799952 5311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:29.799989 5311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d03782a45a74f9c8f0dc99d1d1caaf1, State: Initialized, Role: FOLLOWER
I20260812 01:11:29.800134 5311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:29.800212 5311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:11:29.800241 5311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:11:29.800284 5311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:29.800961 5311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:29.801105 5311 leader_election.cc:302] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d03782a45a74f9c8f0dc99d1d1caaf1; no voters:
I20260812 01:11:29.801331 5311 leader_election.cc:288] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:11:29.801357 5317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:29.801473 5317 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 LEADER]: Becoming Leader. State: Replica: 7d03782a45a74f9c8f0dc99d1d1caaf1, State: Running, Role: LEADER
I20260812 01:11:29.801561 5317 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:29.801621 5311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:29.801898 5317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } } }
I20260812 01:11:29.802073 5317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:29.802307 5317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d03782a45a74f9c8f0dc99d1d1caaf1. Latest consensus state: current_term: 1 leader_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } } }
I20260812 01:11:29.802443 5317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:29.802955 5329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:29.803388 5311 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 01:11:29.803722 5329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 01:11:29 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:29.810631 5329 catalog_manager.cc:1347] Generated new cluster ID: edfea65ab39d4a2fbdf94ebd285e4438
I20260812 01:11:29.810734 5329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:11:29.812331 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:29.797042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47734 (local address 127.17.130.62:44467)
0812 01:11:29.797260 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:29.797270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:29.797307 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:11:29.798145 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:29.798162 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:29.798230 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:29.798400 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:29.799070 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:29.800457 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:29.801546 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:29.802346 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:29.802442 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:29.806536 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:29.806595 (+ 59us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:29.806595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:29.806747 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:29.810246 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:29.810727 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:29.810730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:29.810731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:29.810850 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:29.811126 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:29.811127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:29.811128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:29.811500 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:29.811679 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:29.811807 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 01:11:29.812082 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":97,"threads_started":1}
I20260812 01:11:29.841821 5329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:11:29.842443 5329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:11:29.846930 5329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Generated new TSK 0
I20260812 01:11:29.847227 5329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:11:30.673230 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44467
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:30.774138 5389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:30.774447 5389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:30.774492 5389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:30.775759 5389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:30.775833 5389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:30.775882 5389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:11:30.777628 5389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.17.130.20:46373/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:30.777906 5389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:30.778180 5389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:30.783926 5395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:30.783960 5389 server_base.cc:1034] running on GCE node
W20260812 01:11:30.784103 5399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:30.783897 5397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:30.784464 5389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:30.784819 5389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:30.786013 5389 hybrid_clock.cc:648] HybridClock initialized: now 1786497090785990 us; error 39 us; skew 500 ppm
Aug 12 01:11:30 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497090, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:30.789618 5389 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:11:30.790287 5389 webserver.cc:460] Webserver started at http://127.17.130.1:45885/ using document root <none> and password file <none>
I20260812 01:11:30.790616 5389 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:30.790704 5389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:30.790846 5389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:11:30 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497090, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:30.806888 5389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "472183f8c7094872937fc80c3e559513"
format_stamp: "Formatted at 2026-08-12 01:11:30 on dist-test-slave-tfpf"
server_key: "3beba105210f6b9f6a6b8b7de382fcb2"
server_key_iv: "d07e98d03eba03293d4bffea2c51bd9a"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:30.807421 5389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "472183f8c7094872937fc80c3e559513"
format_stamp: "Formatted at 2026-08-12 01:11:30 on dist-test-slave-tfpf"
server_key: "3beba105210f6b9f6a6b8b7de382fcb2"
server_key_iv: "d07e98d03eba03293d4bffea2c51bd9a"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:30.809926 5389 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260812 01:11:30.825922 5412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:30.826189 5389 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:30.826310 5389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "472183f8c7094872937fc80c3e559513"
format_stamp: "Formatted at 2026-08-12 01:11:30 on dist-test-slave-tfpf"
server_key: "3beba105210f6b9f6a6b8b7de382fcb2"
server_key_iv: "d07e98d03eba03293d4bffea2c51bd9a"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:30.826429 5389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:30.850740 5389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:30.853425 5389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:30.853667 5389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:30.854000 5389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:30.854445 5389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:30.854542 5389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:30.854645 5389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:30.854715 5389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:30.864415 5389 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:41159
I20260812 01:11:30.865095 5389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:11:30.869680 5592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:41159 every 8 connection(s)
I20260812 01:11:30.873942 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 5389
I20260812 01:11:30.874066 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:11:30.874258 17928 external_mini_cluster.cc:1428] Setting key 11c18b2f0b2541b54041a157c9a8d698
Aug 12 01:11:30 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497090, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:30.883662 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:30.866475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:46685 (local address 127.17.130.62:44467)
0812 01:11:30.869210 (+ 2735us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:30.869213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:30.871777 (+ 2564us) server_negotiation.cc:407] Connection header received
0812 01:11:30.872367 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:30.872368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:30.872395 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:30.872468 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:30.873182 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:30.873737 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:30.875162 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:30.875478 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:30.875563 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:30.879310 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:30.879312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:30.879312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:30.879322 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:30.881354 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:30.882058 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:30.882060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:30.882060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:30.882164 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:30.882640 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:30.882641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:30.882641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:30.882841 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:30.882908 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:30.883394 (+ 486us) server_negotiation.cc:299] Negotiation successful
0812 01:11:30.883512 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2704,"thread_start_us":76,"threads_started":1}
I20260812 01:11:30.884387 5598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:30.866466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44467 (local address 127.17.130.1:46685)
0812 01:11:30.871606 (+ 5140us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:30.871652 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:30.872247 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:30.872562 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:30.872566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:30.872792 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:30.873006 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:30.873009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:30.873863 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:30.873864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:30.875063 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:30.875067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:30.875658 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:30.875660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:30.875819 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:30.876600 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:30.876616 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:30.879161 (+ 2545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:30.881471 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:30.881481 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:30.881484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:30.881944 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:30.882242 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:30.882243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:30.882244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:30.882515 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:30.882940 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:30.882948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:30.883175 (+ 227us) client_negotiation.cc:769] Sending connection context
0812 01:11:30.883840 (+ 665us) client_negotiation.cc:240] Negotiation successful
0812 01:11:30.884063 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5086,"thread_start_us":94,"threads_started":1}
I20260812 01:11:30.885056 5595 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44467
I20260812 01:11:30.885186 5595 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:30.885423 5595 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:30.886093 5255 ts_manager.cc:194] Registered new tserver with Master: 472183f8c7094872937fc80c3e559513 (127.17.130.1:41159)
I20260812 01:11:30.887229 5255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:46685
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:11:30.917269 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44467
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:31.050511 5604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:31.050912 5604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:31.051062 5604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:31.053009 5604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:31.053174 5604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:31.053282 5604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:11:31.055734 5604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.17.130.20:46373/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:31.056272 5604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:31.056836 5604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:31.064738 5616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.065410 5613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.065732 5620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:31.066531 5604 server_base.cc:1034] running on GCE node
I20260812 01:11:31.066958 5604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:31.067327 5604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:31.068532 5604 hybrid_clock.cc:648] HybridClock initialized: now 1786497091068500 us; error 45 us; skew 500 ppm
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:31.072736 5604 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:11:31.073487 5604 webserver.cc:460] Webserver started at http://127.17.130.2:40081/ using document root <none> and password file <none>
I20260812 01:11:31.073874 5604 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:31.074024 5604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:31.074229 5604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:31.097316 5604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "24c5a773f60e4873a02190c20c5d8443"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "c3626adb7932c23c8306e8f1260d93d9"
server_key_iv: "02e51e2bebd43598b008fedb019e87e8"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.098013 5604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "24c5a773f60e4873a02190c20c5d8443"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "c3626adb7932c23c8306e8f1260d93d9"
server_key_iv: "02e51e2bebd43598b008fedb019e87e8"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.104888 5604 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:31.124342 5632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.125051 5604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 01:11:31.125201 5604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "24c5a773f60e4873a02190c20c5d8443"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "c3626adb7932c23c8306e8f1260d93d9"
server_key_iv: "02e51e2bebd43598b008fedb019e87e8"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.125303 5604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:31.150460 5604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:31.153211 5604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:31.153465 5604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:31.153793 5604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:31.154300 5604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:31.154409 5604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.154489 5604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:31.154529 5604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.161876 5604 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:43869
I20260812 01:11:31.162534 5604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:11:31.162830 5827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:43869 every 8 connection(s)
I20260812 01:11:31.166446 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 5604
I20260812 01:11:31.166571 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:11:31.166764 17928 external_mini_cluster.cc:1428] Setting key e94840f15318e816a92cc2db0c27b9f3
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:31.174606 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:31.163655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:43149 (local address 127.17.130.62:44467)
0812 01:11:31.163722 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:31.163724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:31.163980 (+ 256us) server_negotiation.cc:407] Connection header received
0812 01:11:31.164754 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:31.164755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:31.164774 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:31.164844 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:31.165667 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.166575 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.168338 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.168679 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.168722 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:31.171413 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:31.171415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:31.171415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:31.171424 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:31.172887 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:31.173417 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:31.173419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:31.173419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:31.173485 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:31.173759 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:31.173759 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:31.173760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:31.173942 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:31.174034 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:31.174364 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 01:11:31.174495 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:11:31.174757 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:31.163612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44467 (local address 127.17.130.2:43149)
0812 01:11:31.163817 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:31.163865 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:31.164637 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:31.164923 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:31.164928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:31.165249 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:31.165551 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:31.165558 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.166716 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.166717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:31.168226 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:31.168230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.168870 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.168870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:31.169019 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:31.169732 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:31.169743 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:31.171305 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:31.173024 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:31.173032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:31.173036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:31.173323 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:31.173554 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:31.173555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:31.173555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:31.173684 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:31.174069 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:31.174076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:31.174218 (+ 142us) client_negotiation.cc:769] Sending connection context
0812 01:11:31.174330 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 01:11:31.174503 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260812 01:11:31.175271 5828 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44467
I20260812 01:11:31.175405 5828 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:31.175624 5828 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:31.176136 5255 ts_manager.cc:194] Registered new tserver with Master: 24c5a773f60e4873a02190c20c5d8443 (127.17.130.2:43869)
I20260812 01:11:31.176550 5255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:43149
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:11:31.204998 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44467
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:31.332927 5836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:31.333262 5836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:31.333303 5836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:31.335202 5836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:31.335273 5836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:31.335315 5836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:11:31.338116 5836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.17.130.20:46373/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:31.338558 5836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:31.338903 5836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:31.346191 5845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.346649 5842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.346792 5847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:31.347762 5836 server_base.cc:1034] running on GCE node
I20260812 01:11:31.348143 5836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:31.348438 5836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:31.349671 5836 hybrid_clock.cc:648] HybridClock initialized: now 1786497091349625 us; error 40 us; skew 500 ppm
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:31.354080 5836 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:11:31.354806 5836 webserver.cc:460] Webserver started at http://127.17.130.3:33577/ using document root <none> and password file <none>
I20260812 01:11:31.355136 5836 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:31.355223 5836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:31.355383 5836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:31.371621 5836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "4d4ebcbb50874469a2db8f03c0025b81"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "9a06cdc90783e78f5afee5bfeeaab617"
server_key_iv: "da3f4828ab5da235eec5cb5066671a2f"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.372165 5836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "4d4ebcbb50874469a2db8f03c0025b81"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "9a06cdc90783e78f5afee5bfeeaab617"
server_key_iv: "da3f4828ab5da235eec5cb5066671a2f"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.374605 5836 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:11:31.395741 5862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.396015 5836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:31.396129 5836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "4d4ebcbb50874469a2db8f03c0025b81"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "9a06cdc90783e78f5afee5bfeeaab617"
server_key_iv: "da3f4828ab5da235eec5cb5066671a2f"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.396229 5836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:31.415849 5836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:31.418435 5836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:31.418672 5836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:31.419186 5836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:31.419629 5836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:31.419704 5836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.419755 5836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:31.419790 5836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.428373 5836 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:46529
I20260812 01:11:31.428999 5836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:11:31.430593 6032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:46529 every 8 connection(s)
I20260812 01:11:31.434751 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 5836
I20260812 01:11:31.434876 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:11:31.435093 17928 external_mini_cluster.cc:1428] Setting key b02ce7e32da9cda570d4cf95c4809c3d
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:31.444502 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:31.431383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:39807 (local address 127.17.130.62:44467)
0812 01:11:31.431433 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:31.431436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:31.431808 (+ 372us) server_negotiation.cc:407] Connection header received
0812 01:11:31.432579 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:31.432581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:31.432599 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:31.432681 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:31.433590 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.434400 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.436635 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.437145 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.437397 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:31.440296 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:31.440298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:31.440298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:31.440308 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:31.442192 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:31.442776 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:31.442777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:31.442777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:31.442848 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:31.443186 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:31.443187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:31.443187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:31.443351 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:31.443435 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:31.444285 (+ 850us) server_negotiation.cc:299] Negotiation successful
0812 01:11:31.444395 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:11:31.444708 6036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:31.431373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44467 (local address 127.17.130.3:39807)
0812 01:11:31.431596 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:31.431694 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:31.432457 (+ 763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:31.432758 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:31.432763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:31.433039 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:31.433394 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:31.433401 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.434536 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.434537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:31.436430 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:31.436434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:31.437423 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:31.437425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:31.437595 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:31.438283 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:31.438294 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:31.440167 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:31.442306 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:31.442316 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:31.442319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:31.442681 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:31.442933 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:31.442934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:31.442935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:31.443097 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:31.443437 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:31.443444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:31.443635 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 01:11:31.444236 (+ 601us) client_negotiation.cc:240] Negotiation successful
0812 01:11:31.444432 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":109,"threads_started":1}
I20260812 01:11:31.445492 6033 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44467
I20260812 01:11:31.445616 6033 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:31.445837 6033 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:31.446507 5255 ts_manager.cc:194] Registered new tserver with Master: 4d4ebcbb50874469a2db8f03c0025b81 (127.17.130.3:46529)
I20260812 01:11:31.447026 5255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:39807
I20260812 01:11:31.448333 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:11:31.449092 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 5389
I20260812 01:11:31.532763 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 5604
I20260812 01:11:31.555893 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 5836
I20260812 01:11:31.572132 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 5198
I20260812 01:11:31.586594 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44467
--webserver_interface=127.17.130.62
--webserver_port=41717
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44467
--ranger_config_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:31.681285 6037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:31.681512 6037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:31.681535 6037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:31.682636 6037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:31.682673 6037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:31.682688 6037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:31.682701 6037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:31.682713 6037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:31.684443 6037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44467
--ranger_config_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=41717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:31.684651 6037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:31.684840 6037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:31.690546 6044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.690526 6045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:31.690539 6037 server_base.cc:1034] running on GCE node
W20260812 01:11:31.690546 6048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:31.691043 6037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:31.691378 6037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:31.692559 6037 hybrid_clock.cc:648] HybridClock initialized: now 1786497091692551 us; error 52 us; skew 500 ppm
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:31.695343 6037 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:31.695842 6037 webserver.cc:460] Webserver started at http://127.17.130.62:41717/ using document root <none> and password file <none>
I20260812 01:11:31.696105 6037 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:31.696161 6037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:31.773921 6037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:11:31.774752 6060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:31.774945 6037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:11:31.775065 6037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1"
format_stamp: "Formatted at 2026-08-12 01:11:29 on dist-test-slave-tfpf"
server_key: "3849ab10496ec6bf84097cfd2c24a7b4"
server_key_iv: "ac8e56cac4dc3dfeabed4f6e5853c9aa"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:31.775460 6037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:31.801599 6037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:31.804498 6037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:31.804692 6037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:31.810467 6037 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44467
I20260812 01:11:31.810508 6142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44467 every 8 connection(s)
I20260812 01:11:31.810935 6037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:31.812611 6143 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:11:31.813386 6143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Bootstrap starting.
I20260812 01:11:31.816742 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 6037
I20260812 01:11:31.817227 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:41159
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:44467
--webserver_port=45885
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:31.822449 6143 log.cc:824] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:31.824318 6143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:11:31.824646 6143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Bootstrap complete.
I20260812 01:11:31.827309 6143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:31.827574 6143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d03782a45a74f9c8f0dc99d1d1caaf1, State: Initialized, Role: FOLLOWER
I20260812 01:11:31.827752 6143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:31.827841 6143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:11:31.827912 6143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:11:31.827967 6143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:11:31.829303 6143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:31.829445 6143 leader_election.cc:302] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d03782a45a74f9c8f0dc99d1d1caaf1; no voters:
I20260812 01:11:31.829670 6143 leader_election.cc:288] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:11:31.829711 6147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:11:31.829888 6147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [term 2 LEADER]: Becoming Leader. State: Replica: 7d03782a45a74f9c8f0dc99d1d1caaf1, State: Running, Role: LEADER
I20260812 01:11:31.830045 6143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:31.830088 6147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } }
I20260812 01:11:31.830379 6147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d03782a45a74f9c8f0dc99d1d1caaf1. Latest consensus state: current_term: 2 leader_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } } }
I20260812 01:11:31.830546 6149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d03782a45a74f9c8f0dc99d1d1caaf1" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44467 } } }
I20260812 01:11:31.830639 6147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:31.830947 6149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:31.831135 6165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:31.831707 6165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:11:31.831916 6165 catalog_manager.cc:1259] Loaded cluster ID: edfea65ab39d4a2fbdf94ebd285e4438
I20260812 01:11:31.832017 6165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:11:31.832126 6143 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 01:11:31.835825 6165 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:11:31.836735 6165 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Loaded TSK: 0
I20260812 01:11:31.837195 6165 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:11:31.955745 6145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:31.956187 6145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:31.956283 6145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:31.958595 6145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:31.958758 6145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:31.958853 6145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:11:31.961467 6145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:41159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=45885
--tserver_master_addrs=127.17.130.62:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:31.962073 6145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:31.962507 6145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:31.969866 6208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.969866 6214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:31.970449 6206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:31.971030 6145 server_base.cc:1034] running on GCE node
I20260812 01:11:31.971312 6145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:31.971658 6145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:31.972868 6145 hybrid_clock.cc:648] HybridClock initialized: now 1786497091972843 us; error 38 us; skew 500 ppm
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:31.976624 6145 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:11:31.977319 6145 webserver.cc:460] Webserver started at http://127.17.130.1:45885/ using document root <none> and password file <none>
I20260812 01:11:31.977638 6145 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:31.977736 6145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:11:31 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:31.997622 6145 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260812 01:11:32.002753 6227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.003222 6145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:32.003360 6145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "472183f8c7094872937fc80c3e559513"
format_stamp: "Formatted at 2026-08-12 01:11:30 on dist-test-slave-tfpf"
server_key: "3beba105210f6b9f6a6b8b7de382fcb2"
server_key_iv: "d07e98d03eba03293d4bffea2c51bd9a"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:32.003790 6145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:32.015631 6145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:32.018656 6145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:32.018844 6145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:32.019155 6145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:32.019594 6145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:32.019660 6145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.019735 6145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:32.019780 6145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.029627 6145 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:41159
I20260812 01:11:32.029750 6407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:41159 every 8 connection(s)
I20260812 01:11:32.030153 6145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:11:32.038363 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 6145
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497091, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:32.038713 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:43869
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:44467
--webserver_port=40081
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:32.043200 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.031253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:37597 (local address 127.17.130.62:44467)
0812 01:11:32.031496 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:32.031506 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:32.031547 (+ 41us) server_negotiation.cc:407] Connection header received
0812 01:11:32.032263 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:32.032271 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:32.032341 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:32.032507 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:32.033365 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.034826 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.036185 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.036621 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.036793 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.039113 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:32.039118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:32.039119 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:32.039130 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:32.041118 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.041689 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.041693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.041693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.041810 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.042123 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.042124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.042130 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.042362 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:32.042461 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:32.042829 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 01:11:32.042985 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":94,"threads_started":1}
I20260812 01:11:32.043215 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.031161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44467 (local address 127.17.130.1:37597)
0812 01:11:32.031346 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:32.031393 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:32.031980 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:32.032587 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:32.032591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:32.032790 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:32.033240 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:32.033248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.035032 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.035033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:32.036074 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:32.036077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.036786 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.036788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:32.036915 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.037531 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:32.037540 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:32.038985 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:32.041228 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:32.041236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:32.041239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:32.041578 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:32.041904 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:32.041905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:32.041905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:32.042038 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:32.042437 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:32.042444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:32.042608 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 01:11:32.042725 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 01:11:32.042904 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260812 01:11:32.044492 6410 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44467
I20260812 01:11:32.044602 6410 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.044801 6410 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
W20260812 01:11:32.045893 6410 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:44467 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 01:11:32.045987 6410 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.046062 6410 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:32.046561 6410 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.046635 6410 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
W20260812 01:11:32.047042 6410 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:11:32.157255 6416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:32.157630 6416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:32.157698 6416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:32.159586 6416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:32.159734 6416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:32.159824 6416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:11:32.162238 6416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:43869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=40081
--tserver_master_addrs=127.17.130.62:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:32.162760 6416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:32.163130 6416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:32.170996 6423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:32.170991 6424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:32.171350 6426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:32.171792 6416 server_base.cc:1034] running on GCE node
I20260812 01:11:32.172057 6416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:32.172366 6416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:32.173588 6416 hybrid_clock.cc:648] HybridClock initialized: now 1786497092173563 us; error 41 us; skew 500 ppm
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497092, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:32.176643 6416 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:11:32.177305 6416 webserver.cc:460] Webserver started at http://127.17.130.2:40081/ using document root <none> and password file <none>
I20260812 01:11:32.177603 6416 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:32.177703 6416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497092, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:32.190713 6416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:32.191956 6440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.192540 6416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:32.192678 6416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "24c5a773f60e4873a02190c20c5d8443"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "c3626adb7932c23c8306e8f1260d93d9"
server_key_iv: "02e51e2bebd43598b008fedb019e87e8"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:32.193109 6416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:32.206110 6416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:32.208963 6416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:32.209167 6416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:32.209440 6416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:32.209793 6416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:32.209846 6416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.209893 6416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:32.209933 6416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.217376 6416 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:43869
I20260812 01:11:32.217489 6621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:43869 every 8 connection(s)
I20260812 01:11:32.217859 6416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:11:32.219045 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 6416
I20260812 01:11:32.219391 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.17.130.20:46373/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:46529
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:44467
--webserver_port=33577
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497092, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:32.230834 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.219278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:57989 (local address 127.17.130.62:44467)
0812 01:11:32.219331 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:32.219333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:32.219603 (+ 270us) server_negotiation.cc:407] Connection header received
0812 01:11:32.220362 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:32.220363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:32.220377 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:32.220413 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:32.221431 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.222201 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.223810 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.224021 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.224136 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.227000 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:32.227003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:32.227003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:32.227013 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:32.228839 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.229465 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.229467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.229467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.229535 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.229912 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.229914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.229914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.230067 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:32.230120 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:32.230571 (+ 451us) server_negotiation.cc:299] Negotiation successful
0812 01:11:32.230705 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:11:32.231719 6626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.219248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44467 (local address 127.17.130.2:57989)
0812 01:11:32.219456 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:32.219509 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:32.220210 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:32.220626 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:32.220643 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:32.220962 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:32.221305 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:32.221312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.222336 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.222337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:32.223644 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:32.223648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.224194 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.224196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:32.224348 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.225008 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:32.225018 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:32.226850 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:32.228954 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:32.228965 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:32.228968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:32.229361 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:32.229675 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:32.229676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:32.229676 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:32.229827 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:32.230154 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:32.230162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:32.230398 (+ 236us) client_negotiation.cc:769] Sending connection context
0812 01:11:32.231217 (+ 819us) client_negotiation.cc:240] Negotiation successful
0812 01:11:32.231444 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260812 01:11:32.232487 6624 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44467
I20260812 01:11:32.232669 6624 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.232949 6624 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
W20260812 01:11:32.233741 6624 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:44467 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 01:11:32.233880 6624 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.233978 6624 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:32.234390 6624 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.234485 6624 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
W20260812 01:11:32.234778 6624 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:11:32.362970 6630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:32.363330 6630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:32.363385 6630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:32.365504 6630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:32.365581 6630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:32.365628 6630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:11:32.368253 6630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.17.130.20:46373/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:46529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=33577
--tserver_master_addrs=127.17.130.62:44467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:32.369225 6630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:32.369733 6630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:32.377398 6639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:32.377430 6641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:32.377740 6638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:32.381371 6630 server_base.cc:1034] running on GCE node
I20260812 01:11:32.381755 6630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:32.382115 6630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:32.383343 6630 hybrid_clock.cc:648] HybridClock initialized: now 1786497092383329 us; error 36 us; skew 500 ppm
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497092, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:32.388263 6630 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:11:32.388953 6630 webserver.cc:460] Webserver started at http://127.17.130.3:33577/ using document root <none> and password file <none>
I20260812 01:11:32.389329 6630 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:32.389506 6630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497092, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:32.412249 6630 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:32.413604 6657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.413847 6630 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:32.413975 6630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "4d4ebcbb50874469a2db8f03c0025b81"
format_stamp: "Formatted at 2026-08-12 01:11:31 on dist-test-slave-tfpf"
server_key: "9a06cdc90783e78f5afee5bfeeaab617"
server_key_iv: "da3f4828ab5da235eec5cb5066671a2f"
server_key_version: "kuduclusterkey@0"
I20260812 01:11:32.414549 6630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:32.449890 6630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:32.452911 6630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:32.453143 6630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:32.453464 6630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:32.453857 6630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:32.453908 6630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.453940 6630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:32.453967 6630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:32.462373 6630 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:46529
I20260812 01:11:32.462942 6630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:11:32.463488 6839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:46529 every 8 connection(s)
I20260812 01:11:32.471601 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 6630
Aug 12 01:11:32 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497092, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:32.477600 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.464640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:60077 (local address 127.17.130.62:44467)
0812 01:11:32.464698 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:32.464701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:32.465143 (+ 442us) server_negotiation.cc:407] Connection header received
0812 01:11:32.466313 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:32.466314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:32.466327 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:32.466388 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:32.467603 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.468157 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.469996 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.470199 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.470283 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.473785 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:32.473787 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:32.473788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:32.473797 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:32.475621 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.476250 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.476252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.476252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.476317 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.476637 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.476638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.476638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.476795 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:32.476851 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:32.477397 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 01:11:32.477494 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 01:11:32.478259 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.464280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44467 (local address 127.17.130.3:60077)
0812 01:11:32.464970 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:32.465021 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:32.466096 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:32.466525 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:32.466531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:32.467008 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:32.467430 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:32.467441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.468376 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.468378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:32.469833 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:32.469838 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.470348 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.470350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:32.470529 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.471319 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:32.471332 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:32.473648 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:32.475731 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:32.475742 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:32.475745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:32.476146 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:32.476387 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:32.476388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:32.476388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:32.476549 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:32.476952 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:32.476962 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:32.477204 (+ 242us) client_negotiation.cc:769] Sending connection context
0812 01:11:32.477775 (+ 571us) client_negotiation.cc:240] Negotiation successful
0812 01:11:32.477974 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":101,"threads_started":1}
I20260812 01:11:32.478855 6842 heartbeater.cc:347] Connected to a master server at 127.17.130.62:44467
I20260812 01:11:32.478976 6842 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.479218 6842 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
W20260812 01:11:32.480188 6842 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:44467 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 01:11:32.480387 6842 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.480518 6842 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:32.481241 6842 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:32.481386 6842 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
W20260812 01:11:32.481788 6842 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:11:32.483129 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:32.472134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47740 (local address 127.17.130.62:44467)
0812 01:11:32.473338 (+ 1204us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:32.473341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:32.473349 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:11:32.473374 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:32.473375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:32.473390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:32.473458 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:32.474329 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.475120 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.476273 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:32.476596 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:32.476665 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:32.479181 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:32.479183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:32.479183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:32.479191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:32.481364 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.481841 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.481842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.481842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.481955 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:32.482215 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:32.482216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:32.482216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:32.482368 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:32.482565 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:32.482640 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 01:11:32.483014 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1182,"thread_start_us":105,"threads_started":1}
I20260812 01:11:33.047222 6410 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:33.047411 6410 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:33.047931 6084 ts_manager.cc:194] Registered new tserver with Master: 472183f8c7094872937fc80c3e559513 (127.17.130.1:41159)
I20260812 01:11:33.048969 6084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:37597
Aug 12 01:11:33 dist-test-slave-tfpf krb5kdc[4489](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786497093, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:33.235005 6624 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:33.235201 6624 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:33.235653 6084 ts_manager.cc:194] Registered new tserver with Master: 24c5a773f60e4873a02190c20c5d8443 (127.17.130.2:43869)
I20260812 01:11:33.236225 6084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:57989
Aug 12 01:11:33 dist-test-slave-tfpf krb5kdc[4489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497093, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for HTTP/127.17.130.20@KRBTEST.COM
I20260812 01:11:33.481990 6842 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:33.482163 6842 heartbeater.cc:511] Master 127.17.130.62:44467 requested a full tablet report, sending...
I20260812 01:11:33.482566 6084 ts_manager.cc:194] Registered new tserver with Master: 4d4ebcbb50874469a2db8f03c0025b81 (127.17.130.3:46529)
I20260812 01:11:33.483081 6084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:60077
I20260812 01:11:33.491110 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:11:33.515336 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.493429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.17.130.62:44467)
0812 01:11:33.493595 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.493597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.497009 (+ 3412us) server_negotiation.cc:407] Connection header received
0812 01:11:33.497104 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.497106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.497126 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.499848 (+ 2722us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:33.501714 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.505498 (+ 3784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.506630 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.506977 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.507057 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.509941 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:33.509943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:33.509943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:33.509953 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:33.512965 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:33.513778 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:33.513780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:33.513780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:33.513856 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:33.514063 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:33.514064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:33.514064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:33.514460 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:33.514572 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.515080 (+ 508us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.515176 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":116,"threads_started":1}
I20260812 01:11:33.518877 6084 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:11:33.521329 6084 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:11:33.532119 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.524670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:41159 (local address 127.0.0.1:37866)
0812 01:11:33.525370 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.525412 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:33.525485 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:33.526179 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:33.526183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:33.526186 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:33.526277 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.526280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.527580 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.527582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.528908 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.528912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.531757 (+ 2845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.531759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.531872 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.531888 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:11:33.531963 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:11:33.532023 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":95,"threads_started":1}
I20260812 01:11:33.536685 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.524923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:46529 (local address 127.0.0.1:38228)
0812 01:11:33.532506 (+ 7583us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.532517 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:33.532563 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:33.532914 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:33.532915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:33.532917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:33.532953 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.532954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.534184 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.534185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.535274 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.535278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.536405 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.536407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.536466 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.536469 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:11:33.536520 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 01:11:33.536570 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7568,"thread_start_us":69,"threads_started":1}
I20260812 01:11:33.537204 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.525158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.17.130.3:46529)
0812 01:11:33.525613 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.525619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.532634 (+ 7015us) server_negotiation.cc:407] Connection header received
0812 01:11:33.532695 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.532698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.532715 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.532788 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:33.533032 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.534079 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.535384 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.536218 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.536295 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.536369 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.537018 (+ 649us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.537117 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":64,"threads_started":1}
I20260812 01:11:33.537987 6732 tablet_service.cc:1467] Processing CreateTablet for tablet 6619305d5b5a4d509aa9a58f2d3a58a3 (DEFAULT_TABLE table=test-table [id=ddb270313de14363a0dc9f51dcf8983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:33.538452 6732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6619305d5b5a4d509aa9a58f2d3a58a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:33.545135 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.524803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37866 (local address 127.17.130.1:41159)
0812 01:11:33.525897 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.525902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.525913 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:11:33.525950 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.525952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.525970 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.526075 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:33.526368 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.527462 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.529143 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.531463 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.531656 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.531728 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.544924 (+ 13196us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.544999 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1058,"thread_start_us":74,"threads_started":1}
I20260812 01:11:33.546082 6296 tablet_service.cc:1467] Processing CreateTablet for tablet 6619305d5b5a4d509aa9a58f2d3a58a3 (DEFAULT_TABLE table=test-table [id=ddb270313de14363a0dc9f51dcf8983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:33.546509 6296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6619305d5b5a4d509aa9a58f2d3a58a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:33.549273 6892 tablet_bootstrap.cc:492] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Bootstrap starting.
I20260812 01:11:33.550050 6892 tablet_bootstrap.cc:654] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:33.550419 6892 log.cc:824] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:33.553464 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.524755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:43869 (local address 127.0.0.1:40128)
0812 01:11:33.525688 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.525698 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:33.525745 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:33.530101 (+ 4356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:33.530102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:33.530106 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:33.530155 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.530157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.550888 (+ 20731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.550890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.552013 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.552017 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.553212 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.553214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.553289 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.553295 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:11:33.553327 (+ 32us) client_negotiation.cc:240] Negotiation successful
0812 01:11:33.553378 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"thread_start_us":64,"threads_started":1}
I20260812 01:11:33.554083 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.525091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.17.130.2:43869)
0812 01:11:33.529412 (+ 4321us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.529418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.529433 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:11:33.529477 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.529479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.529498 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.530240 (+ 742us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:33.530378 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.544783 (+ 14405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.552158 (+ 7375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.553095 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.553846 (+ 751us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.553929 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.553961 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.554004 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3947,"thread_start_us":73,"threads_started":1}
I20260812 01:11:33.554606 6893 tablet_bootstrap.cc:492] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513: Bootstrap starting.
I20260812 01:11:33.554895 6510 tablet_service.cc:1467] Processing CreateTablet for tablet 6619305d5b5a4d509aa9a58f2d3a58a3 (DEFAULT_TABLE table=test-table [id=ddb270313de14363a0dc9f51dcf8983d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:33.555354 6510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6619305d5b5a4d509aa9a58f2d3a58a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:33.555456 6893 tablet_bootstrap.cc:654] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:33.555822 6893 log.cc:824] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:33.556737 6893 tablet_bootstrap.cc:492] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513: No bootstrap required, opened a new log
I20260812 01:11:33.556818 6893 ts_tablet_manager.cc:1397] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:11:33.559255 6893 raft_consensus.cc:348] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.559460 6893 raft_consensus.cc:374] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:33.559506 6893 raft_consensus.cc:729] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 472183f8c7094872937fc80c3e559513, State: Initialized, Role: FOLLOWER
I20260812 01:11:33.559851 6893 consensus_queue.cc:260] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.560195 6893 ts_tablet_manager.cc:1428] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:11:33.560341 6410 heartbeater.cc:503] Master 127.17.130.62:44467 was elected leader, sending a full tablet report...
I20260812 01:11:33.560894 6892 tablet_bootstrap.cc:492] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: No bootstrap required, opened a new log
I20260812 01:11:33.560988 6892 ts_tablet_manager.cc:1397] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260812 01:11:33.563627 6892 raft_consensus.cc:348] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.563824 6892 raft_consensus.cc:374] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:33.563853 6892 raft_consensus.cc:729] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4ebcbb50874469a2db8f03c0025b81, State: Initialized, Role: FOLLOWER
I20260812 01:11:33.564042 6892 consensus_queue.cc:260] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.564414 6892 ts_tablet_manager.cc:1428] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260812 01:11:33.564577 6842 heartbeater.cc:503] Master 127.17.130.62:44467 was elected leader, sending a full tablet report...
I20260812 01:11:33.569811 6896 tablet_bootstrap.cc:492] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Bootstrap starting.
I20260812 01:11:33.570767 6896 tablet_bootstrap.cc:654] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:33.571154 6896 log.cc:824] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:33.574008 6896 tablet_bootstrap.cc:492] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: No bootstrap required, opened a new log
I20260812 01:11:33.574127 6896 ts_tablet_manager.cc:1397] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 01:11:33.577243 6897 raft_consensus.cc:482] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:33.577236 6896 raft_consensus.cc:348] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.577369 6897 raft_consensus.cc:504] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.577440 6896 raft_consensus.cc:374] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:33.577477 6896 raft_consensus.cc:729] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24c5a773f60e4873a02190c20c5d8443, State: Initialized, Role: FOLLOWER
I20260812 01:11:33.577634 6896 consensus_queue.cc:260] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.577754 6897 leader_election.cc:288] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24c5a773f60e4873a02190c20c5d8443 (127.17.130.2:43869), 4d4ebcbb50874469a2db8f03c0025b81 (127.17.130.3:46529)
I20260812 01:11:33.577957 6896 ts_tablet_manager.cc:1428] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:11:33.579028 6624 heartbeater.cc:503] Master 127.17.130.62:44467 was elected leader, sending a full tablet report...
I20260812 01:11:33.586390 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.578618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:33509 (local address 127.17.130.2:43869)
0812 01:11:33.578712 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.578714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.579106 (+ 392us) server_negotiation.cc:407] Connection header received
0812 01:11:33.579267 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.579268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.579277 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.579306 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:33.579544 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.580320 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.585019 (+ 4699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.585694 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.585839 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.585854 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.586250 (+ 396us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.586309 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 01:11:33.587019 6910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.578037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:43869 (local address 127.17.130.1:33509)
0812 01:11:33.579030 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.579042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:33.579182 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:33.579373 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:33.579374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:33.579376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:33.579458 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.579460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.580473 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.580474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.581750 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.581753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.585941 (+ 4188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.585942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.586032 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.586036 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:11:33.586850 (+ 814us) client_negotiation.cc:240] Negotiation successful
0812 01:11:33.586909 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"thread_start_us":97,"threads_started":1}
I20260812 01:11:33.587555 6512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6619305d5b5a4d509aa9a58f2d3a58a3" candidate_uuid: "472183f8c7094872937fc80c3e559513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c5a773f60e4873a02190c20c5d8443" is_pre_election: true
I20260812 01:11:33.587740 6512 raft_consensus.cc:2454] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 472183f8c7094872937fc80c3e559513 in term 0.
I20260812 01:11:33.588092 6229 leader_election.cc:302] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c5a773f60e4873a02190c20c5d8443, 472183f8c7094872937fc80c3e559513; no voters:
I20260812 01:11:33.588248 6897 raft_consensus.cc:2781] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:11:33.588287 6897 raft_consensus.cc:482] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:11:33.588325 6897 raft_consensus.cc:3037] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:33.589324 6897 raft_consensus.cc:504] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.589476 6897 leader_election.cc:288] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [CANDIDATE]: Term 1 election: Requested vote from peers 24c5a773f60e4873a02190c20c5d8443 (127.17.130.2:43869), 4d4ebcbb50874469a2db8f03c0025b81 (127.17.130.3:46529)
I20260812 01:11:33.589818 6512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6619305d5b5a4d509aa9a58f2d3a58a3" candidate_uuid: "472183f8c7094872937fc80c3e559513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24c5a773f60e4873a02190c20c5d8443"
I20260812 01:11:33.589898 6512 raft_consensus.cc:3037] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:33.590802 6512 raft_consensus.cc:2454] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 472183f8c7094872937fc80c3e559513 in term 1.
I20260812 01:11:33.591043 6229 leader_election.cc:302] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24c5a773f60e4873a02190c20c5d8443, 472183f8c7094872937fc80c3e559513; no voters:
I20260812 01:11:33.591177 6897 raft_consensus.cc:2781] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:33.591441 6897 raft_consensus.cc:686] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [term 1 LEADER]: Becoming Leader. State: Replica: 472183f8c7094872937fc80c3e559513, State: Running, Role: LEADER
I20260812 01:11:33.591547 6897 consensus_queue.cc:237] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } }
I20260812 01:11:33.592415 6084 catalog_manager.cc:5495] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 reported cstate change: term changed from 0 to 1, leader changed from <none> to 472183f8c7094872937fc80c3e559513 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "472183f8c7094872937fc80c3e559513" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "472183f8c7094872937fc80c3e559513" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 41159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 } health_report { overall_health: UNKNOWN } } }
I20260812 01:11:33.594815 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.578666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47123 (local address 127.17.130.3:46529)
0812 01:11:33.578748 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.578749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.581958 (+ 3209us) server_negotiation.cc:407] Connection header received
0812 01:11:33.582443 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.582445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.582455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.582483 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:33.582697 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.583536 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.593711 (+ 10175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.594365 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.594427 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.594437 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.594693 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.594740 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 01:11:33.595247 6907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.578292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:46529 (local address 127.17.130.1:47123)
0812 01:11:33.581882 (+ 3590us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.581892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:33.582355 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:33.582547 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:33.582548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:33.582549 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:33.582625 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.582626 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.583659 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.583660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.584892 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:33.584895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.594510 (+ 9615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.594511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:33.594601 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.594605 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:11:33.595086 (+ 481us) client_negotiation.cc:240] Negotiation successful
0812 01:11:33.595144 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3569,"thread_start_us":125,"threads_started":1}
I20260812 01:11:33.595738 6764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6619305d5b5a4d509aa9a58f2d3a58a3" candidate_uuid: "472183f8c7094872937fc80c3e559513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4ebcbb50874469a2db8f03c0025b81" is_pre_election: true
I20260812 01:11:33.595913 6764 raft_consensus.cc:2454] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 472183f8c7094872937fc80c3e559513 in term 0.
I20260812 01:11:33.596086 6762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6619305d5b5a4d509aa9a58f2d3a58a3" candidate_uuid: "472183f8c7094872937fc80c3e559513" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4ebcbb50874469a2db8f03c0025b81"
I20260812 01:11:33.596165 6762 raft_consensus.cc:3037] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:33.597184 6762 raft_consensus.cc:2454] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 472183f8c7094872937fc80c3e559513 in term 1.
I20260812 01:11:33.602875 6882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.599413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37872 (local address 127.17.130.1:41159)
0812 01:11:33.599451 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:33.599453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:33.599460 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:11:33.599494 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:33.599495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:33.599514 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:33.599576 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:11:33.599770 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.600461 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.601627 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:33.601935 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:33.602009 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:33.602231 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:11:33.602369 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:11:33.602455 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:33.602766 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 01:11:33.602794 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:11:33.605268 6762 raft_consensus.cc:1261] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 1 FOLLOWER]: Refusing update from remote peer 472183f8c7094872937fc80c3e559513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:33.605816 6897 consensus_queue.cc:1035] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4ebcbb50874469a2db8f03c0025b81" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:33.608606 6512 raft_consensus.cc:1261] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 1 FOLLOWER]: Refusing update from remote peer 472183f8c7094872937fc80c3e559513: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:33.608963 6897 consensus_queue.cc:1035] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 472183f8c7094872937fc80c3e559513 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24c5a773f60e4873a02190c20c5d8443" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:33.615331 6915 mvcc.cc:205] Tried to move back new op lower bound from 7317492095403618304 to 7317492095351595008. Current Snapshot: MvccSnapshot[applied={T|T < 7317492095403618304}]
I20260812 01:11:33.616109 6916 mvcc.cc:205] Tried to move back new op lower bound from 7317492095403618304 to 7317492095351595008. Current Snapshot: MvccSnapshot[applied={T|T < 7317492095403618304}]
I20260812 01:11:33.619398 6084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:11:33.619560 6084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:11:33.620544 6084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d03782a45a74f9c8f0dc99d1d1caaf1: Sending DeleteTablet for 3 replicas of tablet 6619305d5b5a4d509aa9a58f2d3a58a3
I20260812 01:11:33.621227 6732 tablet_service.cc:1514] Processing DeleteTablet for tablet 6619305d5b5a4d509aa9a58f2d3a58a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:33 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:38228
I20260812 01:11:33.621723 6296 tablet_service.cc:1514] Processing DeleteTablet for tablet 6619305d5b5a4d509aa9a58f2d3a58a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:33 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:37866
I20260812 01:11:33.622021 6510 tablet_service.cc:1514] Processing DeleteTablet for tablet 6619305d5b5a4d509aa9a58f2d3a58a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:33 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:40128
I20260812 01:11:33.622607 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 6145
I20260812 01:11:33.643647 6934 tablet_replica.cc:330] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: stopping tablet replica
I20260812 01:11:33.643839 6934 raft_consensus.cc:2229] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:33.643929 6934 pending_rounds.cc:62] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Trying to abort 2 pending ops.
I20260812 01:11:33.643962 6934 pending_rounds.cc:68] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317492095403618304 op_type: WRITE_OP write_request { tablet_id: "6619305d5b5a4d509aa9a58f2d3a58a3" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\361\222\357\323\006\"8\n\ntest-admin\022*\n ddb270313de14363a0dc9f51dcf8983d\020\001\030\001 \001(\001" signature: "<redacted>""\233\021\323;n\035\210\374\033\244P\320\244\211y\325\377\212]K\005\2712.=\235\341\355\354ZG\310zz\350\251&\255D.gF\010\300\313f\214\372\265\313\316~\342\025\213\263\333\212B\372:\376\357\310" signing_key_seq_num: 0 } } request_id { client_id: "b2a97e1c5a5046e99da5855a056993c8" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 01:11:33.644271 6934 pending_rounds.cc:68] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317492095351595008 op_type: NO_OP noop_request { }
I20260812 01:11:33.644331 6934 raft_consensus.cc:2866] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260812 01:11:33.644366 6934 raft_consensus.cc:2258] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:33.644764 6931 tablet_replica.cc:330] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: stopping tablet replica
I20260812 01:11:33.644867 6931 raft_consensus.cc:2229] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:33.644999 6931 raft_consensus.cc:2258] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:33.645582 6931 ts_tablet_manager.cc:1905] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:33.648479 6931 ts_tablet_manager.cc:1918] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:33.648602 6931 log.cc:1196] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/6619305d5b5a4d509aa9a58f2d3a58a3
I20260812 01:11:33.648957 6931 ts_tablet_manager.cc:1939] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 4d4ebcbb50874469a2db8f03c0025b81: Deleting consensus metadata
I20260812 01:11:33.649574 6063 catalog_manager.cc:4841] TS 4d4ebcbb50874469a2db8f03c0025b81 (127.17.130.3:46529): tablet 6619305d5b5a4d509aa9a58f2d3a58a3 (table test-table [id=ddb270313de14363a0dc9f51dcf8983d]) successfully deleted
I20260812 01:11:33.650015 6934 ts_tablet_manager.cc:1905] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:33.651729 6934 ts_tablet_manager.cc:1918] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:33.651826 6934 log.cc:1196] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/6619305d5b5a4d509aa9a58f2d3a58a3
I20260812 01:11:33.652122 6934 ts_tablet_manager.cc:1939] T 6619305d5b5a4d509aa9a58f2d3a58a3 P 24c5a773f60e4873a02190c20c5d8443: Deleting consensus metadata
I20260812 01:11:33.652762 6062 catalog_manager.cc:4841] TS 24c5a773f60e4873a02190c20c5d8443 (127.17.130.2:43869): tablet 6619305d5b5a4d509aa9a58f2d3a58a3 (table test-table [id=ddb270313de14363a0dc9f51dcf8983d]) successfully deleted
W20260812 01:11:33.660823 6061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:41159 (error 108)
I20260812 01:11:33.661890 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 6416
I20260812 01:11:33.661911 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.661719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:41159 (local address 127.0.0.1:37888)
0812 01:11:33.661787 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.661853 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:41159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 01:11:33.662073 6061 catalog_manager.cc:4584] TS 472183f8c7094872937fc80c3e559513 (127.17.130.1:41159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6619305d5b5a4d509aa9a58f2d3a58a3: Network error: Client connection negotiation failed: client connection to 127.17.130.1:41159: connect: Connection refused (error 111)
I20260812 01:11:33.715116 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.714792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:41159 (local address 127.0.0.1:37890)
0812 01:11:33.714858 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.715048 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:41159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:11:33.715734 6881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:33.715420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:41159 (local address 127.0.0.1:37906)
0812 01:11:33.715650 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:33.715694 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:41159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207}
I20260812 01:11:33.719637 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 6630
I20260812 01:11:33.734596 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 6037
I20260812 01:11:33.755700 17928 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 01:11:34.101387 17928 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 01:11:34.101516 17928 mini_ranger.cc:66] Stopping Ranger...
I20260812 01:11:34.457855 17928 mini_ranger.cc:68] Stopped Ranger
2026-08-12 01:11:34.458 UTC [4530] LOG: received smart shutdown request
2026-08-12 01:11:34.462 UTC [4530] LOG: background worker "logical replication launcher" (PID 4543) exited with exit code 1
2026-08-12 01:11:34.463 UTC [4535] LOG: shutting down
2026-08-12 01:11:34.633 UTC [4530] LOG: database system is shut down
2026-08-12T01:11:34Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55777 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:11:34 dist-test-slave-tfpf krb5kdc[6950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:11:34 dist-test-slave-tfpf krb5kdc[6950](info): set up 2 sockets
Aug 12 01:11:34 dist-test-slave-tfpf krb5kdc[6950](info): commencing operation
krb5kdc: starting...
W20260812 01:11:36.124284 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:11:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:11:36.260207 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44897
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44897,127.17.130.61:42493,127.17.130.60:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:36.348891 6978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:36.349208 6978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:36.349241 6978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:36.350762 6978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:36.350832 6978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:36.350850 6978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:36.350865 6978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:36.350879 6978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:36.352846 6978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44897,127.17.130.61:42493,127.17.130.60:38961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:36.353191 6978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:36.353425 6978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:36.359689 6985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:36.359920 6978 server_base.cc:1034] running on GCE node
W20260812 01:11:36.359669 6988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:36.359932 6986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:36.360723 6978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:36.361135 6978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:36.362319 6978 hybrid_clock.cc:648] HybridClock initialized: now 1786497096362253 us; error 99 us; skew 500 ppm
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:36.365356 6978 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:36.365870 6978 webserver.cc:460] Webserver started at http://127.17.130.62:45855/ using document root <none> and password file <none>
I20260812 01:11:36.366111 6978 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:36.366156 6978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:36.366256 6978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:36.367801 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "0a0aa718ca394ac39f32fdc482a0043c"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "e994df28262219f528b7a5a7cc269463"
server_key_iv: "3577f2f9ea8eab3e4c681c3fba094ab1"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.368216 6978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "0a0aa718ca394ac39f32fdc482a0043c"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "e994df28262219f528b7a5a7cc269463"
server_key_iv: "3577f2f9ea8eab3e4c681c3fba094ab1"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.369843 6978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 01:11:36.370741 6995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:36.371028 6978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:11:36.371121 6978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "0a0aa718ca394ac39f32fdc482a0043c"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "e994df28262219f528b7a5a7cc269463"
server_key_iv: "3577f2f9ea8eab3e4c681c3fba094ab1"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.371224 6978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:36.397734 6978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:36.400240 6978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:36.400399 6978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:36.404947 6978 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44897
I20260812 01:11:36.405143 7077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44897 every 8 connection(s)
I20260812 01:11:36.405406 6978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:36.406385 7079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:36.406600 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 6978
I20260812 01:11:36.406739 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:11:36.407074 17928 external_mini_cluster.cc:1428] Setting key c3bef5020c0833df029d8f8de60cbe49
I20260812 01:11:36.408269 7079 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.61@KRBTEST.COM" created.
I20260812 01:11:36.419873 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.409127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44897 (local address 127.0.0.1:47060)
0812 01:11:36.409344 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.409380 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.409905 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.410209 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.410212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.410515 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.410671 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.410675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.412170 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.412171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.412975 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.412978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.413759 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.413761 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.413829 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.414408 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.414417 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.415788 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.417987 (+ 2199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.417994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.417997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.418199 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.418479 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.418479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.418480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.418637 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.419183 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.419190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.419262 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.419366 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.419719 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":107,"thread_start_us":121,"threads_started":1}
I20260812 01:11:36.419886 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.409371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47060 (local address 127.17.130.62:44897)
0812 01:11:36.409536 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.409540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.409583 (+ 43us) server_negotiation.cc:407] Connection header received
0812 01:11:36.410084 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.410086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.410104 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.410200 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.410786 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.412044 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.413093 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.413655 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.413790 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.415886 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.415895 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.415895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.415903 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.417879 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.418306 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.418309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.418310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.418408 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.418748 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.418749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.418749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.419096 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.419209 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.419404 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.419576 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260812 01:11:36.420826 7079 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:11:36.421264 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.421128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44528)
0812 01:11:36.421190 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.421241 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 01:11:36.421437 6997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
I20260812 01:11:36.421964 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.421837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44536)
0812 01:11:36.421894 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.421930 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 01:11:36.422101 7079 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:42493: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab.
Entry for principal kudu/127.17.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab.
I20260812 01:11:36.436561 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:42493
--webserver_interface=127.17.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab
--principal=kudu/127.17.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44897,127.17.130.61:42493,127.17.130.60:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:36.471366 7079 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } attempt: 1
I20260812 01:11:36.471987 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.471825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44542)
0812 01:11:36.471898 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.471951 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 01:11:36.472720 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.472587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44554)
0812 01:11:36.472654 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.472696 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 01:11:36.472908 7079 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:42493: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
W20260812 01:11:36.529739 7087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:36.530004 7087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:36.530033 7087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:36.531180 7087 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:36.531241 7087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:36.531258 7087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:36.531272 7087 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:36.531286 7087 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:36.532894 7087 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44897,127.17.130.61:42493,127.17.130.60:38961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab
--principal=kudu/127.17.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:42493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:36.533244 7087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:36.533530 7087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:36.538890 7095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:36.538910 7094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:36.538893 7097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:36.539142 7087 server_base.cc:1034] running on GCE node
I20260812 01:11:36.539466 7087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:36.539794 7087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:36.540973 7087 hybrid_clock.cc:648] HybridClock initialized: now 1786497096540933 us; error 57 us; skew 500 ppm
I20260812 01:11:36.541124 7079 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } attempt: 2
I20260812 01:11:36.541613 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.541482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44556)
0812 01:11:36.541546 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.541583 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 01:11:36.542030 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.541922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44570)
0812 01:11:36.541975 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.542008 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260812 01:11:36.542152 7079 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:42493: Network error: Client connection negotiation failed: client connection to 127.17.130.61:42493: connect: Connection refused (error 111)
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:36.543911 7087 init.cc:377] Logged in from keytab as kudu/127.17.130.61@KRBTEST.COM (short username kudu)
I20260812 01:11:36.544410 7087 webserver.cc:460] Webserver started at http://127.17.130.61:43595/ using document root <none> and password file <none>
I20260812 01:11:36.544665 7087 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:36.544725 7087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:36.544863 7087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:36.546025 7087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data/instance:
uuid: "01bb2f367e39438ca475bd056c747288"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "09dd9eb9d97c8ac084e44e14b8bee296"
server_key_iv: "321213e310fd2448dfa305bf267b03fe"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.546412 7087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal/instance:
uuid: "01bb2f367e39438ca475bd056c747288"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "09dd9eb9d97c8ac084e44e14b8bee296"
server_key_iv: "321213e310fd2448dfa305bf267b03fe"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.547837 7087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:11:36.548755 7113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:36.549124 7087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:36.549252 7087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal
uuid: "01bb2f367e39438ca475bd056c747288"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "09dd9eb9d97c8ac084e44e14b8bee296"
server_key_iv: "321213e310fd2448dfa305bf267b03fe"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.549345 7087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:36.580273 7087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:36.582835 7087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:36.583014 7087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:36.587571 7087 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.61:42493
I20260812 01:11:36.587620 7204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.61:42493 every 8 connection(s)
I20260812 01:11:36.588018 7087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
I20260812 01:11:36.588976 7205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:36.590732 7205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:11:36.593659 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 7087
I20260812 01:11:36.593770 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-1/wal/instance
I20260812 01:11:36.593971 17928 external_mini_cluster.cc:1428] Setting key 23f7b493f356a0eaaece643e9294c8bc
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:36.602092 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.591627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.17.130.62:44897)
0812 01:11:36.591761 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.591765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.592004 (+ 239us) server_negotiation.cc:407] Connection header received
0812 01:11:36.592559 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.592560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.592571 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.592632 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.593655 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.594300 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.595800 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.596043 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.596137 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.598774 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.598776 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.598776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.598785 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.600505 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.600966 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.600968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.600968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.601037 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.601352 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.601353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.601353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.601512 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.601602 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.601872 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.601972 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260812 01:11:36.602162 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.591449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44897 (local address 127.0.0.1:47064)
0812 01:11:36.591800 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.591852 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.592417 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.592753 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.592756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.593145 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.593533 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.593541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.594398 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.594400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.595678 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.595681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.596154 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.596156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.596325 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.596943 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.596955 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.598672 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.600619 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.600627 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.600629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.600877 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.601155 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.601156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.601156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.601270 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.601594 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.601600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.601731 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.601819 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.601963 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":118,"threads_started":1}
I20260812 01:11:36.603096 7205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.130.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.60@KRBTEST.COM" created.
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:11:36.612999 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.603382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44574)
0812 01:11:36.603435 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.603446 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.603490 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.603805 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.603806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.603961 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.604010 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.604011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.604898 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.604899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.605471 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.605473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.605991 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.605992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.606078 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.606548 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.606553 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.608112 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.610601 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.610603 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.610603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.610865 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.611432 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.611433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.611433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.611687 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.612570 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.612572 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.612635 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.612730 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.612869 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 01:11:36.613288 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.603463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.17.130.61:42493)
0812 01:11:36.603664 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.603668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.603686 (+ 18us) server_negotiation.cc:407] Connection header received
0812 01:11:36.603704 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.603706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.603716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.603805 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.604090 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.604802 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.605574 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.605869 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.605983 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.608244 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.608256 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.608256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.608264 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.610493 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.611179 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.611182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.611182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.611258 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.612247 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.612248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.612249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.612428 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.612576 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.612781 (+ 205us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.613012 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":172,"thread_start_us":119,"threads_started":1}
I20260812 01:11:36.614127 7205 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:11:36.614598 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.614450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35526)
0812 01:11:36.614523 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.614575 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 01:11:36.614799 7119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
I20260812 01:11:36.615345 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.615244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35540)
0812 01:11:36.615294 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.615323 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:11:36.615490 7205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:38961: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab.
Entry for principal kudu/127.17.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab.
I20260812 01:11:36.627810 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:38961
--webserver_interface=127.17.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab
--principal=kudu/127.17.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44897,127.17.130.61:42493,127.17.130.60:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:36.633309 7079 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } attempt: 3
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:11:36.642966 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.633731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44588)
0812 01:11:36.633798 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.633813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.633870 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.634515 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.634516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.634690 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.634764 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.634766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.635997 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.635999 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.636507 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.636509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.637234 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.637235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.637311 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.637832 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.637838 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.639398 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.641544 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.641545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.641546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.641845 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.642169 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.642170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.642170 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.642262 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.642589 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.642590 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.642644 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.642716 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.642891 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:11:36.642959 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.633833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.17.130.61:42493)
0812 01:11:36.633907 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.633909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.633922 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:11:36.634332 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.634335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.634356 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.634449 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.634861 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.635857 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.636604 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.637134 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.637221 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.639493 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.639494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.639495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.639503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.641420 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.641934 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.641936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.641936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.642003 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.642338 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.642339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.642339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.642489 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.642564 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.642758 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.642850 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:11:36.643817 7079 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:11:36.644284 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.644131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35550)
0812 01:11:36.644224 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.644257 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 01:11:36.644735 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.644602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35554)
0812 01:11:36.644673 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.644711 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 01:11:36.644855 7079 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:38961: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
I20260812 01:11:36.664747 7205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } attempt: 1
I20260812 01:11:36.665390 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.665171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35570)
0812 01:11:36.665239 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.665323 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 01:11:36.665966 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.665806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35582)
0812 01:11:36.665842 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.665940 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 01:11:36.666085 7205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:38961: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
I20260812 01:11:36.676017 7079 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } attempt: 1
I20260812 01:11:36.676538 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.676401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35596)
0812 01:11:36.676466 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.676508 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:11:36.677002 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.676809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35604)
0812 01:11:36.676924 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.676970 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 01:11:36.677134 7079 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:38961: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
W20260812 01:11:36.727260 7213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:36.727665 7213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:36.727716 7213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:36.729650 7213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:36.729735 7213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:36.729761 7213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:36.729784 7213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:36.729815 7213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:36.731608 7213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44897,127.17.130.61:42493,127.17.130.60:38961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab
--principal=kudu/127.17.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:36.731947 7213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:36.732221 7213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:11:36.734225 7205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } attempt: 2
I20260812 01:11:36.734737 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.734577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35618)
0812 01:11:36.734657 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.734703 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 01:11:36.735114 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.735027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35622)
0812 01:11:36.735072 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.735097 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 01:11:36.735236 7205 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:38961: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
W20260812 01:11:36.738005 7223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:36.738056 7228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:36.738005 7225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:36.738268 7213 server_base.cc:1034] running on GCE node
I20260812 01:11:36.738648 7213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:36.738989 7213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:36.740173 7213 hybrid_clock.cc:648] HybridClock initialized: now 1786497096740162 us; error 32 us; skew 500 ppm
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:36.743000 7213 init.cc:377] Logged in from keytab as kudu/127.17.130.60@KRBTEST.COM (short username kudu)
I20260812 01:11:36.743525 7213 webserver.cc:460] Webserver started at http://127.17.130.60:35103/ using document root <none> and password file <none>
I20260812 01:11:36.743798 7213 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:36.743880 7213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:36.744033 7213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:36.745232 7213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data/instance:
uuid: "35a2cb67af8446c192f83ec61f52fa17"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "c565aa9cd89b2e1f168fdf82fa64ed13"
server_key_iv: "5ae55f222c31dbf390bcbbeca720a9cc"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.745608 7213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal/instance:
uuid: "35a2cb67af8446c192f83ec61f52fa17"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "c565aa9cd89b2e1f168fdf82fa64ed13"
server_key_iv: "5ae55f222c31dbf390bcbbeca720a9cc"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.747047 7213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:11:36.747886 7239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:36.748159 7213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:11:36.748298 7213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal
uuid: "35a2cb67af8446c192f83ec61f52fa17"
format_stamp: "Formatted at 2026-08-12 01:11:36 on dist-test-slave-tfpf"
server_key: "c565aa9cd89b2e1f168fdf82fa64ed13"
server_key_iv: "5ae55f222c31dbf390bcbbeca720a9cc"
server_key_version: "encryptionkey@0"
I20260812 01:11:36.748385 7213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:36.752292 7079 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } attempt: 2
I20260812 01:11:36.752743 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.752615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35630)
0812 01:11:36.752673 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.752715 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 01:11:36.753173 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.753022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35636)
0812 01:11:36.753129 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.753153 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260812 01:11:36.753261 7079 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:38961: Network error: Client connection negotiation failed: client connection to 127.17.130.60:38961: connect: Connection refused (error 111)
I20260812 01:11:36.775324 7213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:36.777947 7213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:36.778142 7213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:36.782438 7213 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.60:38961
I20260812 01:11:36.782476 7326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.60:38961 every 8 connection(s)
I20260812 01:11:36.782893 7213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
I20260812 01:11:36.783913 7328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:36.785315 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 7213
I20260812 01:11:36.785436 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786496918848844-17928-0/minicluster-data/master-2/wal/instance
I20260812 01:11:36.785631 17928 external_mini_cluster.cc:1428] Setting key ef4f80b6f2b104353ca5f5a8d04ec739
I20260812 01:11:36.785640 7328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:11:36.798137 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.786433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.17.130.62:44897)
0812 01:11:36.786483 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.786485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.786806 (+ 321us) server_negotiation.cc:407] Connection header received
0812 01:11:36.787303 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.787304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.787318 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.787383 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.788656 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.789459 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.791019 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.791334 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.791416 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.794260 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.794262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.794262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.794274 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.796050 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.796760 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.796761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.796761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.796827 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.797183 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.797185 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.797185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.797388 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.797478 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.797782 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.798009 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 01:11:36.798151 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.786339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44897 (local address 127.0.0.1:47078)
0812 01:11:36.786477 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.786619 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.787176 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.787439 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.787442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.787830 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.788373 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.788390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.789737 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.789738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.790892 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.790895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.791441 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.791442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.791564 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.792334 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.792344 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.794148 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.796153 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.796162 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.796166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.796654 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.796906 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.796907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.796908 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.797042 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.797474 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.797480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.797617 (+ 137us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.797753 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.797930 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260812 01:11:36.798863 7328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:11:36.799021 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.786215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.17.130.60:38961)
0812 01:11:36.786425 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.786434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.786468 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:11:36.787047 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.787056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.787082 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.787194 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.788645 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.789784 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.790884 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.791495 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.791563 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.795228 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.795236 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.795236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.795244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.797325 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.797698 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.797701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.797701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.797766 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.798096 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.798097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.798097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.798270 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.798423 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.798748 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.798830 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:36.805706 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.799203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:42493 (local address 127.0.0.1:44604)
0812 01:11:36.799241 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.799252 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.799290 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.799438 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.799439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.799607 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.799661 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.799663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.800329 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.800331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.800837 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.800839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.801270 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.801271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.801341 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.801766 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.801770 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.802958 (+ 1188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.804635 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.804636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.804636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.804807 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.805019 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.805020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.805020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.805110 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.805367 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.805368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.805410 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.805491 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.805605 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 01:11:36.805706 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.799270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.17.130.61:42493)
0812 01:11:36.799316 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.799318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.799326 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:11:36.799356 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.799357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.799365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.799425 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.799746 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.800248 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.800929 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.801190 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.801261 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.803054 (+ 1793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.803056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.803056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.803064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.804537 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.804889 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.804890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.804893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.804948 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.805170 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.805171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.805172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.805286 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.805361 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.805533 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.805607 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:11:36.806056 7328 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:11:36.807578 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.799862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47092 (local address 127.17.130.62:44897)
0812 01:11:36.799899 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.799901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.799944 (+ 43us) server_negotiation.cc:407] Connection header received
0812 01:11:36.800015 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.800016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.800025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.800086 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.800556 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.801016 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.801904 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.802117 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.802204 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.804681 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.804683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.804683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.804691 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.806385 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.806740 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.806741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.806742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.806799 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.807019 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.807020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.807020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.807136 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.807318 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.807356 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.807456 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:11:36.813712 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.806347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35650)
0812 01:11:36.806386 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.806396 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.806431 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.806533 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.806534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.806672 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.806697 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.806698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.807301 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.807302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.807977 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.807980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.808440 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.808441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.808499 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.808908 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.808912 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.810504 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.812433 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.812434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.812434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.812686 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.812929 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.812930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.812930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.813082 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.813390 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.813392 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.813456 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.813543 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.813640 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 01:11:36.813834 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.806412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.17.130.60:38961)
0812 01:11:36.806448 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.806450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.806457 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:11:36.806478 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.806478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.806486 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.806532 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.806769 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.807222 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.808052 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.808366 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.808435 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.810608 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.810610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.810610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.810617 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.812337 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.812786 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.812787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.812788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.812845 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.813185 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.813186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.813186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.813314 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.813373 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.813595 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.813713 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":20}
I20260812 01:11:36.816241 7328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: Bootstrap starting.
I20260812 01:11:36.817617 7328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:36.817857 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.808257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44616 (local address 127.17.130.61:42493)
0812 01:11:36.808294 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.808296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.808303 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:11:36.808323 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.808323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.808333 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.808381 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.809010 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.809669 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.810643 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.810989 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.811079 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.814073 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.814075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.814075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.814085 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.815956 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.816339 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.816340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.816341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.816410 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.816612 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.816613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.816613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.816784 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.817609 (+ 825us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.817660 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.817748 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:11:36.817981 7328 log.cc:824] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:36.818902 7328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: No bootstrap required, opened a new log
I20260812 01:11:36.819943 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:11:36.820861 7328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:36.821110 7328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:36.821175 7328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a2cb67af8446c192f83ec61f52fa17, State: Initialized, Role: FOLLOWER
I20260812 01:11:36.821336 7328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:36.821712 7328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:36.821730 7348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:36.821861 7348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:11:36.826383 7205 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } attempt: 3
I20260812 01:11:36.824601 7213 master_runner.cc:186] Error getting master registration for 127.17.130.62:44897: OK, instance_id { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" instance_seqno: 1786497096403616 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:11:36.827454 7213 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 01:11:36.827673 7377 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:11:36.829332 7377 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:11:36.831066 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.821582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.17.130.61:42493)
0812 01:11:36.821620 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.821622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.822089 (+ 467us) server_negotiation.cc:407] Connection header received
0812 01:11:36.822186 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.822187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.822199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.822229 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.822901 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.823642 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.824937 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.825280 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.825323 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.827626 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.827628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.827628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.827636 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.829665 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.830081 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.830082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.830083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.830152 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.830434 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.830435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.830435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.830593 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.830721 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.830839 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.830948 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:11:36.831888 7331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.821428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.17.130.60:38961)
0812 01:11:36.821469 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.821471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.822067 (+ 596us) server_negotiation.cc:407] Connection header received
0812 01:11:36.822180 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.822181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.822193 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.822228 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.823124 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.823760 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.824852 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.825200 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.825248 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.827465 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.827466 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.827467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.827474 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.829464 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.830746 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.830747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.830747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.830828 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.831127 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.831128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.831129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.831321 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.831419 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.831710 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.831795 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:11:36.833719 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.821616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47094 (local address 127.17.130.62:44897)
0812 01:11:36.821673 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.821676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.821900 (+ 224us) server_negotiation.cc:407] Connection header received
0812 01:11:36.821993 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.821994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.822006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.822063 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.822900 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.823651 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.824715 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.825007 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.825045 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.827993 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.827994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.827994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.828001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.830195 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.831941 (+ 1746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.831943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.831943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.832014 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.832343 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.832344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.832344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.832486 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.832572 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.833409 (+ 837us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.833511 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:11:36.835947 7207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.826634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35664)
0812 01:11:36.826886 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.826898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.826945 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.827821 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.827822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.827985 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.828029 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.828031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.829009 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.829011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.829851 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.829854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.830395 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.830396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.830461 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.831057 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.831062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.832603 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.834749 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.834750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.834751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.834947 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.835207 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.835208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.835208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.835275 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.835557 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.835558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.835627 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.835721 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.835835 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260812 01:11:36.835973 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.827040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.17.130.60:38961)
0812 01:11:36.827675 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.827677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.827684 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:11:36.827704 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.827705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.827716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.828112 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.828146 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.828909 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.829952 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.830309 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.830372 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.832723 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.832724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.832725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.832732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.834646 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.835042 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.835043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.835044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.835133 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.835352 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.835353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.835353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.835481 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.835565 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.835769 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.835849 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":102,"threads_started":1}
I20260812 01:11:36.837945 7205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: Bootstrap starting.
I20260812 01:11:36.838864 7205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:36.839195 7205 log.cc:824] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:36.840247 7205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: No bootstrap required, opened a new log
I20260812 01:11:36.842607 7205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:36.842775 7205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:36.842816 7205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01bb2f367e39438ca475bd056c747288, State: Initialized, Role: FOLLOWER
I20260812 01:11:36.842941 7205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:36.843287 7382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:36.843328 7205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:36.843415 7382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:11:36.845731 7403 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:11:36.845801 7403 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:11:36.845880 7087 master_runner.cc:186] Error getting master registration for 127.17.130.62:44897: OK, instance_id { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" instance_seqno: 1786497096403616 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:11:36.849366 7087 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:11:36.852409 7079 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } attempt: 3
Aug 12 01:11:36 dist-test-slave-tfpf krb5kdc[6950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497096, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:11:36.862357 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.852826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:38961 (local address 127.0.0.1:35680)
0812 01:11:36.852886 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:36.852898 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:36.852969 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:36.853188 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:36.853189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:36.853404 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:36.853467 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.853469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.854493 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.854495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.855261 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:36.855264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.856116 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.856120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:36.856233 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.856935 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:36.856942 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:36.858654 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:36.860764 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.860766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.860766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.861258 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.861543 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:36.861544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:36.861545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:36.861634 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:36.862019 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:36.862020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:36.862076 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 01:11:36.862144 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:11:36.862257 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:11:36.863191 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:36.852875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.17.130.60:38961)
0812 01:11:36.852932 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:36.852935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:36.852956 (+ 21us) server_negotiation.cc:407] Connection header received
0812 01:11:36.853050 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:36.853051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:36.853087 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:36.853165 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:36.853563 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.854401 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.855539 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:36.855860 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:36.855954 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:36.858763 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:36.858765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:36.858765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:36.858773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:36.860653 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.861355 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.861357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.861357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.861429 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:36.861719 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:36.861720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:36.861721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:36.861905 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:36.861985 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:36.862994 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0812 01:11:36.863097 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:11:36.865227 7079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: Bootstrap starting.
I20260812 01:11:36.866022 7079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:36.866349 7079 log.cc:824] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:36.867326 7079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: No bootstrap required, opened a new log
I20260812 01:11:36.869477 7079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:36.869647 7079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:36.869683 7079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a0aa718ca394ac39f32fdc482a0043c, State: Initialized, Role: FOLLOWER
I20260812 01:11:36.869812 7079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:36.870158 7079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:36.870126 7406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:36.870332 7406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:11:36.871939 7420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:11:36.872051 7420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:11:36.875298 6978 master_runner.cc:263] No leader master found from master 0a0aa718ca394ac39f32fdc482a0043c
I20260812 01:11:36.875361 6978 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 01:11:36.901321 7004 tablet.cc:2369] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:11:37.028857 7250 tablet.cc:2369] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:11:37.083961 7122 tablet.cc:2369] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:11:37.108513 7382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:37.108672 7382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:37.109144 7382 leader_election.cc:288] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a0aa718ca394ac39f32fdc482a0043c (127.17.130.62:44897), 35a2cb67af8446c192f83ec61f52fa17 (127.17.130.60:38961)
I20260812 01:11:37.109512 7286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01bb2f367e39438ca475bd056c747288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a2cb67af8446c192f83ec61f52fa17" is_pre_election: true
I20260812 01:11:37.109542 7035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01bb2f367e39438ca475bd056c747288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0aa718ca394ac39f32fdc482a0043c" is_pre_election: true
I20260812 01:11:37.109699 7035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bb2f367e39438ca475bd056c747288 in term 0.
I20260812 01:11:37.109674 7286 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01bb2f367e39438ca475bd056c747288 in term 0.
I20260812 01:11:37.109935 7119 leader_election.cc:302] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bb2f367e39438ca475bd056c747288, 35a2cb67af8446c192f83ec61f52fa17; no voters:
I20260812 01:11:37.110087 7382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:11:37.110157 7382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:11:37.110193 7382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:37.111022 7382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:37.111187 7382 leader_election.cc:288] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [CANDIDATE]: Term 1 election: Requested vote from peers 0a0aa718ca394ac39f32fdc482a0043c (127.17.130.62:44897), 35a2cb67af8446c192f83ec61f52fa17 (127.17.130.60:38961)
I20260812 01:11:37.111410 7035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01bb2f367e39438ca475bd056c747288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a0aa718ca394ac39f32fdc482a0043c"
I20260812 01:11:37.111421 7286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01bb2f367e39438ca475bd056c747288" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35a2cb67af8446c192f83ec61f52fa17"
I20260812 01:11:37.111514 7035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:37.111511 7286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:37.112242 7035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bb2f367e39438ca475bd056c747288 in term 1.
I20260812 01:11:37.112424 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01bb2f367e39438ca475bd056c747288, 0a0aa718ca394ac39f32fdc482a0043c; no voters:
I20260812 01:11:37.112533 7382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:37.112692 7382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [term 1 LEADER]: Becoming Leader. State: Replica: 01bb2f367e39438ca475bd056c747288, State: Running, Role: LEADER
I20260812 01:11:37.112689 7286 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01bb2f367e39438ca475bd056c747288 in term 1.
I20260812 01:11:37.112790 7382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } }
I20260812 01:11:37.113230 7424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01bb2f367e39438ca475bd056c747288. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.113346 7424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:37.113579 7428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:37.113796 7428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:11:37.115375 7035 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [term 1 FOLLOWER]: Refusing update from remote peer 01bb2f367e39438ca475bd056c747288: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:37.115443 7286 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [term 1 FOLLOWER]: Refusing update from remote peer 01bb2f367e39438ca475bd056c747288: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:37.115743 7382 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:37.115854 7424 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:37.117081 7406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01bb2f367e39438ca475bd056c747288. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.117187 7406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:11:37.117487 7348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01bb2f367e39438ca475bd056c747288. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.117587 7348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:11:37.118088 7424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.118180 7424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:37.118172 7382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.118284 7382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:37.118333 7434 mvcc.cc:205] Tried to move back new op lower bound from 7317492109781741568 to 7317492109774618624. Current Snapshot: MvccSnapshot[applied={T|T < 7317492109781741568}]
I20260812 01:11:37.118407 7436 mvcc.cc:205] Tried to move back new op lower bound from 7317492109781741568 to 7317492109774618624. Current Snapshot: MvccSnapshot[applied={T|T < 7317492109781741568}]
I20260812 01:11:37.118392 7406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.118461 7406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:11:37.118474 7348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01bb2f367e39438ca475bd056c747288" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a0aa718ca394ac39f32fdc482a0043c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44897 } } peers { permanent_uuid: "01bb2f367e39438ca475bd056c747288" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 42493 } } peers { permanent_uuid: "35a2cb67af8446c192f83ec61f52fa17" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 38961 } } }
I20260812 01:11:37.118551 7348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:11:37.119055 7428 catalog_manager.cc:1347] Generated new cluster ID: 391afc2aad374b54a05bb05aad4f6e10
I20260812 01:11:37.119107 7428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:11:37.132207 7428 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:11:37.133016 7428 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:11:37.138893 7428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01bb2f367e39438ca475bd056c747288: Generated new TSK 0
I20260812 01:11:37.139149 7428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:11:37.830070 7377 catalog_manager.cc:1259] Loaded cluster ID: 391afc2aad374b54a05bb05aad4f6e10
I20260812 01:11:37.830207 7377 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: loading cluster ID for follower catalog manager: success
I20260812 01:11:37.831061 7377 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: acquiring CA information for follower catalog manager: success
I20260812 01:11:37.831416 7377 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 35a2cb67af8446c192f83ec61f52fa17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:11:37.873618 7420 catalog_manager.cc:1259] Loaded cluster ID: 391afc2aad374b54a05bb05aad4f6e10
I20260812 01:11:37.873760 7420 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: loading cluster ID for follower catalog manager: success
I20260812 01:11:37.874681 7420 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: acquiring CA information for follower catalog manager: success
I20260812 01:11:37.875066 7420 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0a0aa718ca394ac39f32fdc482a0043c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:11:38.193025 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 6978
I20260812 01:11:38.207661 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 7087
I20260812 01:11:38.224610 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 7213
2026-08-12T01:11:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3462 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:11:38 dist-test-slave-tfpf krb5kdc[7462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:11:38 dist-test-slave-tfpf krb5kdc[7462](info): set up 2 sockets
Aug 12 01:11:38 dist-test-slave-tfpf krb5kdc[7462](info): commencing operation
krb5kdc: starting...
W20260812 01:11:39.562381 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:11:39 dist-test-slave-tfpf krb5kdc[7462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:11:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:11:39.708621 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:42043
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:42043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:39.805163 7483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:39.805446 7483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:39.805487 7483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:39.806756 7483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:39.807355 7483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:39.807432 7483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:39.807467 7483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:39.807492 7483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:39.809463 7483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:42043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:42043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:39.809851 7483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:39.810138 7483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:39.816174 7492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:39.816159 7497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:39.816159 7489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:39.816412 7483 server_base.cc:1034] running on GCE node
I20260812 01:11:39.816785 7483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:39.817198 7483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:39.818400 7483 hybrid_clock.cc:648] HybridClock initialized: now 1786497099818342 us; error 78 us; skew 500 ppm
Aug 12 01:11:39 dist-test-slave-tfpf krb5kdc[7462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497099, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:39.821375 7483 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:39.821956 7483 webserver.cc:460] Webserver started at http://127.17.130.62:33439/ using document root <none> and password file <none>
I20260812 01:11:39.822227 7483 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:39.822285 7483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:39.822453 7483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:39.824247 7483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "c5a87a50491140608b47306115971cd7"
format_stamp: "Formatted at 2026-08-12 01:11:39 on dist-test-slave-tfpf"
server_key: "aa1febfbdd93e491c7bb5039a0bcd748"
server_key_iv: "0cf62838d011e431c38cb0894f5f16f4"
server_key_version: "encryptionkey@0"
I20260812 01:11:39.824855 7483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "c5a87a50491140608b47306115971cd7"
format_stamp: "Formatted at 2026-08-12 01:11:39 on dist-test-slave-tfpf"
server_key: "aa1febfbdd93e491c7bb5039a0bcd748"
server_key_iv: "0cf62838d011e431c38cb0894f5f16f4"
server_key_version: "encryptionkey@0"
I20260812 01:11:39.833509 7483 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.000s
I20260812 01:11:39.835183 7507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:39.835449 7483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:11:39.835605 7483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "c5a87a50491140608b47306115971cd7"
format_stamp: "Formatted at 2026-08-12 01:11:39 on dist-test-slave-tfpf"
server_key: "aa1febfbdd93e491c7bb5039a0bcd748"
server_key_iv: "0cf62838d011e431c38cb0894f5f16f4"
server_key_version: "encryptionkey@0"
I20260812 01:11:39.835709 7483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:39.878815 7483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:39.882292 7483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:39.882522 7483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:39.887399 7483 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:42043
I20260812 01:11:39.887450 7611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:42043 every 8 connection(s)
I20260812 01:11:39.887923 7483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:39.889168 7612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:39.893399 7612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7: Bootstrap starting.
I20260812 01:11:39.894393 7612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:39.894786 7612 log.cc:824] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:39.895680 7612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7: No bootstrap required, opened a new log
I20260812 01:11:39.897624 7612 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a87a50491140608b47306115971cd7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42043 } }
I20260812 01:11:39.897795 7612 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:39.897818 7612 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5a87a50491140608b47306115971cd7, State: Initialized, Role: FOLLOWER
I20260812 01:11:39.897992 7612 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a87a50491140608b47306115971cd7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42043 } }
I20260812 01:11:39.898095 7612 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:11:39.898092 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 7483
I20260812 01:11:39.898141 7612 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:11:39.898193 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:11:39.898200 7612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:39.898432 17928 external_mini_cluster.cc:1428] Setting key 8035c1d1f7b9cebbed917a138a96fd62
I20260812 01:11:39.899103 7612 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a87a50491140608b47306115971cd7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42043 } }
I20260812 01:11:39.899238 7612 leader_election.cc:302] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5a87a50491140608b47306115971cd7; no voters:
I20260812 01:11:39.899410 7612 leader_election.cc:288] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:11:39.899554 7622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:39.899749 7612 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:39.899789 7622 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [term 1 LEADER]: Becoming Leader. State: Replica: c5a87a50491140608b47306115971cd7, State: Running, Role: LEADER
I20260812 01:11:39.900112 7622 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a87a50491140608b47306115971cd7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42043 } }
I20260812 01:11:39.900625 7622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5a87a50491140608b47306115971cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a87a50491140608b47306115971cd7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42043 } } }
I20260812 01:11:39.900777 7622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:39.900961 7622 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5a87a50491140608b47306115971cd7. Latest consensus state: current_term: 1 leader_uuid: "c5a87a50491140608b47306115971cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5a87a50491140608b47306115971cd7" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42043 } } }
I20260812 01:11:39.901124 7622 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7 [sys.catalog]: This master's current role is: LEADER
W20260812 01:11:39.901912 7642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:11:39.902025 7642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:11:39.903116 7635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:39.903682 7635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:11:39.905584 7635 catalog_manager.cc:1347] Generated new cluster ID: bc49e9c0e55f410bb0f1a81d0c47cc90
I20260812 01:11:39.905639 7635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:11:39 dist-test-slave-tfpf krb5kdc[7462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497099, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:39.914413 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:39.899000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.17.130.62:42043)
0812 01:11:39.899280 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:39.899290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:39.899332 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:11:39.899520 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:39.899528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:39.899597 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:39.900735 (+ 1138us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:39.900840 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:39.902296 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:39.903502 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:39.904330 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:39.904825 (+ 495us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:39.908524 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:39.908529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:39.908530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:39.908547 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:39.911698 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:39.912265 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:39.912269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:39.912270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:39.912419 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:39.912661 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:39.912673 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:39.912674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:39.913099 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:39.913840 (+ 741us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:39.913931 (+ 91us) server_negotiation.cc:299] Negotiation successful
0812 01:11:39.914111 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":136,"threads_started":1}
I20260812 01:11:39.915913 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:11:39.925356 7635 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:11:39.926182 7635 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:11:39.941164 7635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5a87a50491140608b47306115971cd7: Generated new TSK 0
I20260812 01:11:39.941412 7635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:11:39.971221 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 7483
2026-08-12T01:11:39Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1744 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:11:40 dist-test-slave-tfpf krb5kdc[7650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:11:40 dist-test-slave-tfpf krb5kdc[7650](info): set up 2 sockets
Aug 12 01:11:40 dist-test-slave-tfpf krb5kdc[7650](info): commencing operation
krb5kdc: starting...
W20260812 01:11:41.325928 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:11:41 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:11:41.473414 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40367
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:40367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:41.570585 7676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:41.570822 7676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:41.570847 7676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:41.572057 7676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:41.572114 7676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:41.572135 7676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:41.572149 7676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:41.572161 7676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:41.573782 7676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:40367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:41.574030 7676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:41.574265 7676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:41.579998 7690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:41.580035 7676 server_base.cc:1034] running on GCE node
W20260812 01:11:41.579993 7686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:41.579996 7687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:41.580595 7676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:41.581146 7676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:41.582387 7676 hybrid_clock.cc:648] HybridClock initialized: now 1786497101582363 us; error 40 us; skew 500 ppm
Aug 12 01:11:41 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:41.585394 7676 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:41.585953 7676 webserver.cc:460] Webserver started at http://127.17.130.62:44039/ using document root <none> and password file <none>
I20260812 01:11:41.586236 7676 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:41.586309 7676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:41.586468 7676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:41.587625 7676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "466e199d2f00460c90442b73f005210f"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "ef58460c6f4379d313e98a5b1f632a18"
server_key_iv: "82552d78ce7c4c1dfd1536e3546e361a"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.588032 7676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "466e199d2f00460c90442b73f005210f"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "ef58460c6f4379d313e98a5b1f632a18"
server_key_iv: "82552d78ce7c4c1dfd1536e3546e361a"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.589632 7676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:11:41.590552 7707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:41.590785 7676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:41.590909 7676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "466e199d2f00460c90442b73f005210f"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "ef58460c6f4379d313e98a5b1f632a18"
server_key_iv: "82552d78ce7c4c1dfd1536e3546e361a"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.590996 7676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:41.613807 7676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:41.616519 7676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:41.616820 7676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:41.622486 7676 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:40367
I20260812 01:11:41.622543 7803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:40367 every 8 connection(s)
I20260812 01:11:41.622956 7676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:41.624158 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:41.626748 7804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap starting.
I20260812 01:11:41.627413 7804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:41.627743 7804 log.cc:824] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:41.628783 7804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: No bootstrap required, opened a new log
I20260812 01:11:41.630537 7804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:41.630687 7804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:41.630712 7804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466e199d2f00460c90442b73f005210f, State: Initialized, Role: FOLLOWER
I20260812 01:11:41.630875 7804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:41.631127 7804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:11:41.631201 7804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:11:41.631264 7804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:41.632439 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 7676
I20260812 01:11:41.632570 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:11:41.632793 17928 external_mini_cluster.cc:1428] Setting key c5726c26456953f939c3a07135490032
I20260812 01:11:41.632829 7804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:41.633090 7804 leader_election.cc:302] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 466e199d2f00460c90442b73f005210f; no voters:
I20260812 01:11:41.633356 7804 leader_election.cc:288] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:11:41.633378 7812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:41.633602 7812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 LEADER]: Becoming Leader. State: Replica: 466e199d2f00460c90442b73f005210f, State: Running, Role: LEADER
I20260812 01:11:41.634037 7812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:41.634047 7804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:41.635159 7815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "466e199d2f00460c90442b73f005210f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } } }
I20260812 01:11:41.635327 7815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:41.635594 7813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 466e199d2f00460c90442b73f005210f. Latest consensus state: current_term: 1 leader_uuid: "466e199d2f00460c90442b73f005210f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } } }
I20260812 01:11:41.635735 7813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:41.635890 7824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:41.636430 7824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:11:41.638926 7824 catalog_manager.cc:1347] Generated new cluster ID: 5163b671954747a2a6b40e34730ad003
I20260812 01:11:41.638988 7824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:11:41 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:41.645386 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:41.633521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54990 (local address 127.17.130.62:40367)
0812 01:11:41.633693 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:41.633703 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:41.634164 (+ 461us) server_negotiation.cc:407] Connection header received
0812 01:11:41.634305 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:41.634314 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:41.634374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:41.634531 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:41.635354 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:41.637240 (+ 1886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:41.638074 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:41.638674 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:41.638814 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:41.641464 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:41.641468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:41.641468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:41.641482 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:41.643439 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:41.643941 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:41.643945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:41.643946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:41.644095 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:41.644349 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:41.644350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:41.644350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:41.644639 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:41.644846 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:41.644926 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 01:11:41.645138 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
I20260812 01:11:41.648041 7824 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:11:41.648643 7824 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:11:41.654438 7824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Generated new TSK 0
I20260812 01:11:41.654717 7824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:11:41.692920 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40367
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:41.805871 7847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:41.806124 7847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:41.806167 7847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:41.807600 7847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:41.807704 7847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:41.807760 7847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:11:41.809731 7847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:41.810045 7847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:41.810329 7847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:41.816073 7860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:41.816154 7864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:41.816145 7859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:41.816417 7847 server_base.cc:1034] running on GCE node
I20260812 01:11:41.816773 7847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:41.817047 7847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:41.818252 7847 hybrid_clock.cc:648] HybridClock initialized: now 1786497101818217 us; error 56 us; skew 500 ppm
Aug 12 01:11:41 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:41.821252 7847 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:11:41.821811 7847 webserver.cc:460] Webserver started at http://127.17.130.1:45829/ using document root <none> and password file <none>
I20260812 01:11:41.822077 7847 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:41.822124 7847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:41.822220 7847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:41.823308 7847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "3fb37b12da984ee299b7a3a2efade141"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "1db89fe49b6a03d3ccfa79d755717faa"
server_key_iv: "1da4fab3584fe4d50c8e77275793df0b"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.823642 7847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "3fb37b12da984ee299b7a3a2efade141"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "1db89fe49b6a03d3ccfa79d755717faa"
server_key_iv: "1da4fab3584fe4d50c8e77275793df0b"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.825371 7847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:11:41.826256 7872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:41.826496 7847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:11:41.826615 7847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "3fb37b12da984ee299b7a3a2efade141"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "1db89fe49b6a03d3ccfa79d755717faa"
server_key_iv: "1da4fab3584fe4d50c8e77275793df0b"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.826718 7847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:41.838409 7847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:41.841531 7847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:41.841712 7847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:41.841992 7847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:41.842367 7847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:41.842423 7847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:41.842464 7847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:41.842495 7847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:41.848452 7847 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:34481
I20260812 01:11:41.848539 8066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:34481 every 8 connection(s)
I20260812 01:11:41.848886 7847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:11:41.852098 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 7847
I20260812 01:11:41.852217 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:11:41.852434 17928 external_mini_cluster.cc:1428] Setting key 3792b5ceb14029f9e6d053fd7f5b5580
Aug 12 01:11:41 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:41.860253 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:41.849947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:57925 (local address 127.17.130.62:40367)
0812 01:11:41.850056 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:41.850058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:41.850244 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:11:41.850765 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:41.850766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:41.850777 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:41.850850 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:41.851861 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:41.852497 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:41.853908 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:41.854123 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:41.854207 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:41.856849 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:41.856851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:41.856851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:41.856861 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:41.858685 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:41.859135 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:41.859144 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:41.859145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:41.859216 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:41.859481 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:41.859482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:41.859482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:41.859650 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:41.859734 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:41.860000 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 01:11:41.860102 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 01:11:41.860277 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:41.849893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:57925)
0812 01:11:41.850081 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:41.850144 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:41.850653 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:41.850908 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:41.850911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:41.851522 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:41.851741 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:41.851746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:41.852610 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:41.852612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:41.853813 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:41.853817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:41.854219 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:41.854220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:41.854365 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:41.854940 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:41.854965 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:41.856741 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:41.858788 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:41.858796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:41.858798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:41.859051 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:41.859295 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:41.859295 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:41.859296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:41.859402 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:41.859726 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:41.859732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:41.859857 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 01:11:41.859945 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 01:11:41.860090 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260812 01:11:41.860867 8068 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40367
I20260812 01:11:41.860988 8068 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:41.861248 8068 heartbeater.cc:511] Master 127.17.130.62:40367 requested a full tablet report, sending...
I20260812 01:11:41.861882 7735 ts_manager.cc:194] Registered new tserver with Master: 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481)
I20260812 01:11:41.862528 7735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:57925
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:11:41.884323 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40367
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:41.976989 8081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:41.977303 8081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:41.977356 8081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:41.978731 8081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:41.978822 8081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:41.978873 8081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:11:41.980623 8081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:41.980969 8081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:41.981417 8081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:41.988178 8088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:41.988415 8089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:41.988778 8093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:41.988973 8081 server_base.cc:1034] running on GCE node
I20260812 01:11:41.989313 8081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:41.989686 8081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:41.990911 8081 hybrid_clock.cc:648] HybridClock initialized: now 1786497101990872 us; error 67 us; skew 500 ppm
Aug 12 01:11:41 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:41.994534 8081 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:11:41.995131 8081 webserver.cc:460] Webserver started at http://127.17.130.2:46777/ using document root <none> and password file <none>
I20260812 01:11:41.995388 8081 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:41.995462 8081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:41.995620 8081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:41.996819 8081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "0fe9991259cf44a6981b05c9cac2c08b"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "6e3cea571b38648ede14f93d6f670417"
server_key_iv: "5ebb231dc5fec4aee32243c445099360"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.997362 8081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "0fe9991259cf44a6981b05c9cac2c08b"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "6e3cea571b38648ede14f93d6f670417"
server_key_iv: "5ebb231dc5fec4aee32243c445099360"
server_key_version: "encryptionkey@0"
I20260812 01:11:41.999235 8081 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 01:11:42.000209 8101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.000478 8081 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:42.000610 8081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "0fe9991259cf44a6981b05c9cac2c08b"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "6e3cea571b38648ede14f93d6f670417"
server_key_iv: "5ebb231dc5fec4aee32243c445099360"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.000699 8081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:42.013355 8081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:42.016685 8081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:42.016830 8081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:42.017102 8081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:42.017580 8081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:42.017622 8081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.017663 8081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:42.017678 8081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.025817 8081 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:44709
I20260812 01:11:42.025861 8303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:44709 every 8 connection(s)
I20260812 01:11:42.026372 8081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:11:42.031577 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 8081
I20260812 01:11:42.031708 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:11:42.031942 17928 external_mini_cluster.cc:1428] Setting key 4416c07d31124ea4f43ed317454d2e3d
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497101, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:42.038753 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.027302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:47741 (local address 127.17.130.62:40367)
0812 01:11:42.027475 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.027477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.027536 (+ 59us) server_negotiation.cc:407] Connection header received
0812 01:11:42.028109 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.028111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.028125 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.028164 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:42.029296 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.029812 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.030912 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.031115 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.031208 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.034617 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:42.034619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:42.034619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:42.034631 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:42.036527 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.037091 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.037093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.037093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.037168 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.037575 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.037577 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.037578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.037830 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:42.037955 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.038425 (+ 470us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.038660 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260812 01:11:42.038900 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.027211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:47741)
0812 01:11:42.027401 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.027450 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.027996 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.028244 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.028248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.028898 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:42.029188 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.029194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.029920 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.029921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.030813 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.030815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.031218 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.031219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.031355 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.032097 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:42.032109 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:42.034468 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:42.036658 (+ 2190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.036669 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.036684 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.036966 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.037254 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.037255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.037256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.037417 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.037968 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:42.037977 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:42.038197 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.038348 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.038573 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260812 01:11:42.039594 8304 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40367
I20260812 01:11:42.039731 8304 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:42.039950 8304 heartbeater.cc:511] Master 127.17.130.62:40367 requested a full tablet report, sending...
I20260812 01:11:42.040798 7735 ts_manager.cc:194] Registered new tserver with Master: 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709)
I20260812 01:11:42.041462 7735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:47741
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:11:42.066013 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40367
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:42.158615 8312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:42.158849 8312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:42.158876 8312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:42.160127 8312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:42.160181 8312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:42.160210 8312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:11:42.161912 8312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:42.162200 8312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:42.162436 8312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:42.168380 8319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.168409 8322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.168412 8318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:42.169463 8312 server_base.cc:1034] running on GCE node
I20260812 01:11:42.169732 8312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:42.170073 8312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:42.171269 8312 hybrid_clock.cc:648] HybridClock initialized: now 1786497102171240 us; error 55 us; skew 500 ppm
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:42.174177 8312 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:11:42.174716 8312 webserver.cc:460] Webserver started at http://127.17.130.3:37121/ using document root <none> and password file <none>
I20260812 01:11:42.175004 8312 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:42.175058 8312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:42.175208 8312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:11:42.176401 8312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "af62d748810f4c78b457d64c4facebcb"
format_stamp: "Formatted at 2026-08-12 01:11:42 on dist-test-slave-tfpf"
server_key: "e6dccfa7080f9aca5b08aa136f349707"
server_key_iv: "118b39cc48ec07b87a78e5e2f9efe842"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.176872 8312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "af62d748810f4c78b457d64c4facebcb"
format_stamp: "Formatted at 2026-08-12 01:11:42 on dist-test-slave-tfpf"
server_key: "e6dccfa7080f9aca5b08aa136f349707"
server_key_iv: "118b39cc48ec07b87a78e5e2f9efe842"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.178678 8312 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 01:11:42.180092 8336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.180361 8312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:42.180488 8312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "af62d748810f4c78b457d64c4facebcb"
format_stamp: "Formatted at 2026-08-12 01:11:42 on dist-test-slave-tfpf"
server_key: "e6dccfa7080f9aca5b08aa136f349707"
server_key_iv: "118b39cc48ec07b87a78e5e2f9efe842"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.180588 8312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:42.194834 8312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:42.198823 8312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:42.199095 8312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:42.199472 8312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:42.200151 8312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:42.200271 8312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.200316 8312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:42.200336 8312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.208436 8312 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:33609
I20260812 01:11:42.208559 8532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:33609 every 8 connection(s)
I20260812 01:11:42.208973 8312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:11:42.213104 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 8312
I20260812 01:11:42.213212 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:11:42.213394 17928 external_mini_cluster.cc:1428] Setting key ccf6e58d2225b0e071228039451ebd2d
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:42.221217 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.209990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:44257 (local address 127.17.130.62:40367)
0812 01:11:42.210114 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.210117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.210338 (+ 221us) server_negotiation.cc:407] Connection header received
0812 01:11:42.210850 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.210852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.210866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.210954 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:42.212041 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.212559 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.214092 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.214369 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.214463 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.217380 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:42.217382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:42.217382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:42.217391 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:42.219092 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.220027 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.220028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.220029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.220099 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.220450 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.220451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.220451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.220586 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:42.220652 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.220945 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.221036 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 01:11:42.221481 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.209915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:44257)
0812 01:11:42.210118 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.210167 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.210721 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.210994 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.210997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.211641 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:42.211868 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.211873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.212680 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.212681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.213980 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.213984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.214477 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.214478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.214647 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.215604 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:42.215619 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:42.217273 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:42.219298 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.219325 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.219328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.219894 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.220217 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.220219 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.220219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.220369 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.220659 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:42.220665 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:42.220809 (+ 144us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.220909 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.221159 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260812 01:11:42.222296 8535 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40367
I20260812 01:11:42.222486 8535 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:42.222757 8535 heartbeater.cc:511] Master 127.17.130.62:40367 requested a full tablet report, sending...
I20260812 01:11:42.223457 7735 ts_manager.cc:194] Registered new tserver with Master: af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
I20260812 01:11:42.223861 7735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:44257
I20260812 01:11:42.226284 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:11:42.234851 7816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.227790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.17.130.62:40367)
0812 01:11:42.227931 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.227935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.228123 (+ 188us) server_negotiation.cc:407] Connection header received
0812 01:11:42.228316 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.228318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.228343 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.228435 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:42.229122 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.229620 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.230423 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.230620 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.230729 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.232060 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:42.232062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:42.232062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:42.232070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:42.233555 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.233934 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.233935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.233936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.234003 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.234239 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.234240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.234241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.234411 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:42.234529 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.234640 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.234726 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 01:11:42.237517 7735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:11:42.239285 7735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:11:42.248157 8559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.242950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34481 (local address 127.0.0.1:48862)
0812 01:11:42.243159 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.243201 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.243273 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.243776 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.243779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.243782 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:42.243873 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.243875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.245342 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.245343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.246740 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.246744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.247797 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.247799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.247894 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.247914 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.247969 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.248034 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":51,"thread_start_us":90,"threads_started":1}
I20260812 01:11:42.248355 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.242966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:44709 (local address 127.0.0.1:59512)
0812 01:11:42.243399 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.243408 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.243445 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.243873 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.243874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.243875 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:42.243911 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.243912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.245342 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.245343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.246774 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.246776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.247799 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.247800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.248196 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.248201 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.248238 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.248285 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":20,"thread_start_us":87,"threads_started":1}
I20260812 01:11:42.248643 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.243126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.17.130.1:34481)
0812 01:11:42.243614 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.243620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.243631 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:11:42.243676 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.243678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.243695 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.243770 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:42.243988 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.245203 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.246859 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.247681 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.248396 (+ 715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.248481 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.248513 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.248565 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":96,"threads_started":1}
I20260812 01:11:42.248693 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.243226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.17.130.2:44709)
0812 01:11:42.243640 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.243645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.243655 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:11:42.243697 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.243699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.243715 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.243798 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:42.244008 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.245214 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.246902 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.247680 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.248463 (+ 783us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.248527 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.248551 (+ 24us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.248610 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":128,"threads_started":1}
I20260812 01:11:42.249441 8188 tablet_service.cc:1467] Processing CreateTablet for tablet 36b61daaf6cf402b942a5efed12f9604 (DEFAULT_TABLE table=test-table [id=11820c70cb0d4e0d90e03b17daebf141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:42.249714 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.243415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.17.130.3:33609)
0812 01:11:42.244060 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.244064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.244071 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:11:42.244102 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.244104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.244115 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.244206 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:42.244331 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.245045 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.247084 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.249199 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.249268 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.249324 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.249573 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.249626 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":93,"threads_started":1}
I20260812 01:11:42.249923 8188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36b61daaf6cf402b942a5efed12f9604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:42.249725 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.243191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:33609 (local address 127.0.0.1:32972)
0812 01:11:42.243451 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.243488 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.243553 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.244205 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.244206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.244208 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:42.244259 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.244261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.245216 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.245217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.246935 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.246939 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.249370 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.249371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.249448 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.249451 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.249513 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.249563 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":124,"thread_start_us":74,"threads_started":1}
I20260812 01:11:42.249616 7951 tablet_service.cc:1467] Processing CreateTablet for tablet 36b61daaf6cf402b942a5efed12f9604 (DEFAULT_TABLE table=test-table [id=11820c70cb0d4e0d90e03b17daebf141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:42.250622 7951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36b61daaf6cf402b942a5efed12f9604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:42.251686 8410 tablet_service.cc:1467] Processing CreateTablet for tablet 36b61daaf6cf402b942a5efed12f9604 (DEFAULT_TABLE table=test-table [id=11820c70cb0d4e0d90e03b17daebf141]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:42.252192 8410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 36b61daaf6cf402b942a5efed12f9604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:42.254333 8570 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap starting.
I20260812 01:11:42.255105 8571 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Bootstrap starting.
I20260812 01:11:42.255424 8570 tablet_bootstrap.cc:654] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:42.256182 8573 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Bootstrap starting.
I20260812 01:11:42.256193 8571 tablet_bootstrap.cc:654] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:42.256717 8570 log.cc:824] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:42.256827 8573 tablet_bootstrap.cc:654] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:42.257090 8571 log.cc:824] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:42.257556 8573 log.cc:824] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:42.257826 8570 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: No bootstrap required, opened a new log
I20260812 01:11:42.257918 8570 ts_tablet_manager.cc:1397] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 01:11:42.258037 8571 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: No bootstrap required, opened a new log
I20260812 01:11:42.258143 8571 ts_tablet_manager.cc:1397] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:11:42.258410 8573 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: No bootstrap required, opened a new log
I20260812 01:11:42.258496 8573 ts_tablet_manager.cc:1397] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:11:42.259774 8570 raft_consensus.cc:348] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.259958 8570 raft_consensus.cc:374] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:42.260005 8570 raft_consensus.cc:729] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb37b12da984ee299b7a3a2efade141, State: Initialized, Role: FOLLOWER
I20260812 01:11:42.260125 8570 consensus_queue.cc:260] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.260464 8570 ts_tablet_manager.cc:1428] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:11:42.260480 8068 heartbeater.cc:503] Master 127.17.130.62:40367 was elected leader, sending a full tablet report...
I20260812 01:11:42.260303 8571 raft_consensus.cc:348] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.260581 8571 raft_consensus.cc:374] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:42.260627 8571 raft_consensus.cc:729] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe9991259cf44a6981b05c9cac2c08b, State: Initialized, Role: FOLLOWER
I20260812 01:11:42.260744 8571 consensus_queue.cc:260] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.260653 8573 raft_consensus.cc:348] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.260825 8573 raft_consensus.cc:374] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:42.260878 8573 raft_consensus.cc:729] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af62d748810f4c78b457d64c4facebcb, State: Initialized, Role: FOLLOWER
I20260812 01:11:42.261030 8571 ts_tablet_manager.cc:1428] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:11:42.261000 8573 consensus_queue.cc:260] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.261315 8573 ts_tablet_manager.cc:1428] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260812 01:11:42.261329 8535 heartbeater.cc:503] Master 127.17.130.62:40367 was elected leader, sending a full tablet report...
I20260812 01:11:42.261397 8304 heartbeater.cc:503] Master 127.17.130.62:40367 was elected leader, sending a full tablet report...
W20260812 01:11:42.277256 8307 tablet.cc:2369] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:11:42.341710 8583 raft_consensus.cc:482] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:42.341864 8583 raft_consensus.cc:504] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.342471 8583 leader_election.cc:288] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
I20260812 01:11:42.346139 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.342710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34481 (local address 127.17.130.2:44579)
0812 01:11:42.342767 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.342780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.342835 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.343227 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.343228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.343230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:42.343310 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.343311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.344039 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.344040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.344943 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.344945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.345751 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.345752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.345853 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.345857 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.345976 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.346040 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 01:11:42.346151 8561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.342939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:44579 (local address 127.17.130.1:34481)
0812 01:11:42.343061 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.343064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.343076 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:11:42.343115 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.343116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.343127 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.343199 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:42.343387 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.343928 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.345122 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.345654 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.345752 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.345765 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.346024 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.346074 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 01:11:42.346625 7989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36b61daaf6cf402b942a5efed12f9604" candidate_uuid: "0fe9991259cf44a6981b05c9cac2c08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb37b12da984ee299b7a3a2efade141" is_pre_election: true
I20260812 01:11:42.346846 7989 raft_consensus.cc:2454] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fe9991259cf44a6981b05c9cac2c08b in term 0.
I20260812 01:11:42.346987 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.343037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:59349 (local address 127.17.130.3:33609)
0812 01:11:42.343096 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.343098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.343138 (+ 40us) server_negotiation.cc:407] Connection header received
0812 01:11:42.343390 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.343391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.343399 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.343558 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:42.343968 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.344530 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.345973 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.346577 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.346676 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.346685 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.346881 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.346923 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 01:11:42.347020 8585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.342881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:33609 (local address 127.17.130.2:59349)
0812 01:11:42.343061 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.343073 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.343150 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.343472 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.343473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.343474 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:42.343711 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.343718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.344647 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.344648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.345857 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.345860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.346676 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.346677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.346795 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.346801 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.346865 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.346913 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":128,"threads_started":1}
I20260812 01:11:42.347191 8103 leader_election.cc:302] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe9991259cf44a6981b05c9cac2c08b, 3fb37b12da984ee299b7a3a2efade141; no voters:
I20260812 01:11:42.347340 8583 raft_consensus.cc:2781] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:11:42.347409 8583 raft_consensus.cc:482] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:11:42.347442 8583 raft_consensus.cc:3037] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:42.347632 8449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36b61daaf6cf402b942a5efed12f9604" candidate_uuid: "0fe9991259cf44a6981b05c9cac2c08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af62d748810f4c78b457d64c4facebcb" is_pre_election: true
I20260812 01:11:42.347802 8449 raft_consensus.cc:2454] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fe9991259cf44a6981b05c9cac2c08b in term 0.
I20260812 01:11:42.348802 8583 raft_consensus.cc:504] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.349025 8583 leader_election.cc:288] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [CANDIDATE]: Term 1 election: Requested vote from peers 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
I20260812 01:11:42.349516 8449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36b61daaf6cf402b942a5efed12f9604" candidate_uuid: "0fe9991259cf44a6981b05c9cac2c08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af62d748810f4c78b457d64c4facebcb"
I20260812 01:11:42.349455 7989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "36b61daaf6cf402b942a5efed12f9604" candidate_uuid: "0fe9991259cf44a6981b05c9cac2c08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fb37b12da984ee299b7a3a2efade141"
I20260812 01:11:42.349669 8449 raft_consensus.cc:3037] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:42.349718 7989 raft_consensus.cc:3037] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Advancing to term 1
W20260812 01:11:42.349819 8072 tablet.cc:2369] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:11:42.350796 7989 raft_consensus.cc:2454] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fe9991259cf44a6981b05c9cac2c08b in term 1.
I20260812 01:11:42.350808 8449 raft_consensus.cc:2454] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fe9991259cf44a6981b05c9cac2c08b in term 1.
I20260812 01:11:42.351050 8103 leader_election.cc:302] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe9991259cf44a6981b05c9cac2c08b, 3fb37b12da984ee299b7a3a2efade141; no voters:
I20260812 01:11:42.351186 8583 raft_consensus.cc:2781] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:42.351370 8583 raft_consensus.cc:686] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 LEADER]: Becoming Leader. State: Replica: 0fe9991259cf44a6981b05c9cac2c08b, State: Running, Role: LEADER
I20260812 01:11:42.351487 8583 consensus_queue.cc:237] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.352777 7735 catalog_manager.cc:5495] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2). New cstate: current_term: 1 leader_uuid: "0fe9991259cf44a6981b05c9cac2c08b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } health_report { overall_health: UNKNOWN } } }
I20260812 01:11:42.367470 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.363848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.17.130.2:44709)
0812 01:11:42.363982 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.363987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.364100 (+ 113us) server_negotiation.cc:407] Connection header received
0812 01:11:42.364294 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.364295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.364325 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.364426 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:11:42.364618 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.365279 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.366191 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.366442 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.366549 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.366878 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:11:42.367023 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:11:42.367185 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.367237 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.367276 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 01:11:42.370281 8449 raft_consensus.cc:1261] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Refusing update from remote peer 0fe9991259cf44a6981b05c9cac2c08b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:42.370281 7989 raft_consensus.cc:1261] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Refusing update from remote peer 0fe9991259cf44a6981b05c9cac2c08b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:42.371114 8583 consensus_queue.cc:1035] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [LEADER]: Connected to new peer: Peer: permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:42.371498 8587 consensus_queue.cc:1035] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:42.375918 8594 mvcc.cc:205] Tried to move back new op lower bound from 7317492131304116224 to 7317492131232567296. Current Snapshot: MvccSnapshot[applied={T|T < 7317492131304116224 or (T in {7317492131304116224})}]
I20260812 01:11:42.376921 8599 mvcc.cc:205] Tried to move back new op lower bound from 7317492131304116224 to 7317492131232567296. Current Snapshot: MvccSnapshot[applied={T|T < 7317492131304116224}]
I20260812 01:11:42.377435 8595 mvcc.cc:205] Tried to move back new op lower bound from 7317492131304116224 to 7317492131232567296. Current Snapshot: MvccSnapshot[applied={T|T < 7317492131304116224}]
I20260812 01:11:42.378211 7735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:11:42.378464 7735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:11:42.379487 7735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Sending DeleteTablet for 3 replicas of tablet 36b61daaf6cf402b942a5efed12f9604
I20260812 01:11:42.379912 8188 tablet_service.cc:1514] Processing DeleteTablet for tablet 36b61daaf6cf402b942a5efed12f9604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:42 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:59512
I20260812 01:11:42.379971 7951 tablet_service.cc:1514] Processing DeleteTablet for tablet 36b61daaf6cf402b942a5efed12f9604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:42 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:48862
I20260812 01:11:42.380015 8410 tablet_service.cc:1514] Processing DeleteTablet for tablet 36b61daaf6cf402b942a5efed12f9604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:42 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:32972
I20260812 01:11:42.380234 8613 tablet_replica.cc:330] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: stopping tablet replica
I20260812 01:11:42.380328 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 7847
I20260812 01:11:42.380362 8613 raft_consensus.cc:2229] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:42.387939 8612 tablet_replica.cc:330] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: stopping tablet replica
I20260812 01:11:42.388135 8612 raft_consensus.cc:2229] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:11:42.388302 8612 raft_consensus.cc:2258] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:42.388624 8614 tablet_replica.cc:330] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: stopping tablet replica
I20260812 01:11:42.388797 8614 raft_consensus.cc:2229] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:42.388968 8614 raft_consensus.cc:2258] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:42.389590 8612 ts_tablet_manager.cc:1905] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:42.392107 8612 ts_tablet_manager.cc:1918] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:42.392383 8612 log.cc:1196] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/36b61daaf6cf402b942a5efed12f9604
I20260812 01:11:42.392418 8614 ts_tablet_manager.cc:1905] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:42.393167 8612 ts_tablet_manager.cc:1939] T 36b61daaf6cf402b942a5efed12f9604 P 0fe9991259cf44a6981b05c9cac2c08b: Deleting consensus metadata
I20260812 01:11:42.393929 7710 catalog_manager.cc:4841] TS 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709): tablet 36b61daaf6cf402b942a5efed12f9604 (table test-table [id=11820c70cb0d4e0d90e03b17daebf141]) successfully deleted
I20260812 01:11:42.395766 8614 ts_tablet_manager.cc:1918] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:42.395876 8614 log.cc:1196] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/36b61daaf6cf402b942a5efed12f9604
I20260812 01:11:42.396240 8614 ts_tablet_manager.cc:1939] T 36b61daaf6cf402b942a5efed12f9604 P af62d748810f4c78b457d64c4facebcb: Deleting consensus metadata
I20260812 01:11:42.396986 7708 catalog_manager.cc:4841] TS af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609): tablet 36b61daaf6cf402b942a5efed12f9604 (table test-table [id=11820c70cb0d4e0d90e03b17daebf141]) successfully deleted
W20260812 01:11:42.401037 7708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:34481 (error 108)
I20260812 01:11:42.401746 8560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.401590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34481 (local address 127.0.0.1:48866)
0812 01:11:42.401658 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.401702 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:11:42.401849 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 8081
W20260812 01:11:42.410614 7708 catalog_manager.cc:4584] TS 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 36b61daaf6cf402b942a5efed12f9604: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34481: connect: Connection refused (error 111)
I20260812 01:11:42.419957 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 8312
I20260812 01:11:42.444896 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 7676
I20260812 01:11:42.466539 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40367
--webserver_interface=127.17.130.62
--webserver_port=44039
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:40367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:42.565251 8618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:42.565498 8618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:42.565528 8618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:42.566941 8618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:42.566998 8618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:42.567014 8618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:42.567026 8618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:42.567039 8618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:42.568667 8618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:40367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=44039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:42.569010 8618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:42.569306 8618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:42.581424 8625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.581436 8626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.581604 8632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:42.582397 8618 server_base.cc:1034] running on GCE node
I20260812 01:11:42.582777 8618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:42.583179 8618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:42.584388 8618 hybrid_clock.cc:648] HybridClock initialized: now 1786497102584347 us; error 89 us; skew 500 ppm
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:42.587539 8618 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:42.588128 8618 webserver.cc:460] Webserver started at http://127.17.130.62:44039/ using document root <none> and password file <none>
I20260812 01:11:42.588394 8618 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:42.588495 8618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:42.590209 8618 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:42.591358 8643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.591840 8618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:42.592074 8618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "466e199d2f00460c90442b73f005210f"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "ef58460c6f4379d313e98a5b1f632a18"
server_key_iv: "82552d78ce7c4c1dfd1536e3546e361a"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.592572 8618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:42.611294 8618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:42.614503 8618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:42.614765 8618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:42.620630 8618 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:40367
I20260812 01:11:42.620667 8744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:40367 every 8 connection(s)
I20260812 01:11:42.621205 8618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:42.623018 8746 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:11:42.623790 8746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap starting.
I20260812 01:11:42.625053 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 8618
I20260812 01:11:42.625474 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:34481
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:40367
--webserver_port=45829
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:42.638404 8746 log.cc:824] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:42.640707 8746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:11:42.641036 8746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap complete.
I20260812 01:11:42.643725 8746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:42.644006 8746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466e199d2f00460c90442b73f005210f, State: Initialized, Role: FOLLOWER
I20260812 01:11:42.644220 8746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:42.644313 8746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:11:42.644372 8746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:11:42.644420 8746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:11:42.645655 8746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:42.645792 8746 leader_election.cc:302] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 466e199d2f00460c90442b73f005210f; no voters:
I20260812 01:11:42.646062 8746 leader_election.cc:288] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:11:42.646116 8751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:11:42.646243 8751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 LEADER]: Becoming Leader. State: Replica: 466e199d2f00460c90442b73f005210f, State: Running, Role: LEADER
I20260812 01:11:42.646353 8751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:42.646389 8746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:42.646852 8751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 466e199d2f00460c90442b73f005210f. Latest consensus state: current_term: 2 leader_uuid: "466e199d2f00460c90442b73f005210f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } } }
I20260812 01:11:42.646974 8751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:42.647346 8763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:42.646849 8753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "466e199d2f00460c90442b73f005210f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } } }
I20260812 01:11:42.647670 8753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:42.648331 8763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:11:42.648443 8763 catalog_manager.cc:1259] Loaded cluster ID: 5163b671954747a2a6b40e34730ad003
I20260812 01:11:42.648468 8763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:11:42.648718 8763 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 01:11:42.653717 8763 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 01:11:42.656810 8763 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:11:42.657128 8763 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Loaded TSK: 0
I20260812 01:11:42.657394 8763 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:11:42.725943 8749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:42.726203 8749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:42.726262 8749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:42.727571 8749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:42.727648 8749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:42.727703 8749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:11:42.729336 8749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:34481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=45829
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:42.729629 8749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:42.729907 8749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:42.736723 8796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.736843 8801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.736928 8797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:42.737222 8749 server_base.cc:1034] running on GCE node
I20260812 01:11:42.737468 8749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:42.737752 8749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:42.738970 8749 hybrid_clock.cc:648] HybridClock initialized: now 1786497102738942 us; error 43 us; skew 500 ppm
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:42.741863 8749 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:11:42.742372 8749 webserver.cc:460] Webserver started at http://127.17.130.1:45829/ using document root <none> and password file <none>
I20260812 01:11:42.742632 8749 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:42.742734 8749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:42.744318 8749 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:11:42.745302 8809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.745597 8749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:42.745741 8749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "3fb37b12da984ee299b7a3a2efade141"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "1db89fe49b6a03d3ccfa79d755717faa"
server_key_iv: "1da4fab3584fe4d50c8e77275793df0b"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.746201 8749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:42.763425 8749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:42.766434 8749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:42.766628 8749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:42.766914 8749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:42.767565 8818 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:11:42.769711 8749 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:11:42.769893 8749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 01:11:42.769950 8749 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:11:42.770671 8749 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:11:42.770731 8749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 01:11:42.770812 8818 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap starting.
I20260812 01:11:42.779531 8749 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:34481
I20260812 01:11:42.779611 8994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:34481 every 8 connection(s)
I20260812 01:11:42.780118 8749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:11:42.782940 8818 log.cc:824] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:42.784215 8818 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 01:11:42.784611 8818 tablet_bootstrap.cc:492] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap complete.
I20260812 01:11:42.785295 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 8749
I20260812 01:11:42.785843 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:44709
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:40367
--webserver_port=46777
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:42.788744 8818 ts_tablet_manager.cc:1397] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.002s
I20260812 01:11:42.790445 8818 raft_consensus.cc:348] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.791520 8818 raft_consensus.cc:729] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb37b12da984ee299b7a3a2efade141, State: Initialized, Role: FOLLOWER
I20260812 01:11:42.791740 8818 consensus_queue.cc:260] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } } peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } }
I20260812 01:11:42.792093 8818 ts_tablet_manager.cc:1428] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:42.799423 8998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.780987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:44147)
0812 01:11:42.781319 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.781371 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.782043 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.785016 (+ 2973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.785021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.785695 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:42.786004 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.786013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.787868 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.787870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.790130 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.790134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.791339 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.791341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.791488 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.792204 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:42.792214 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:42.793758 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:42.796657 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.796690 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.796693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.797265 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.797703 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.797704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.797705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.798033 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.798659 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:42.798668 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:42.798818 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.798933 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.799193 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":125,"threads_started":1}
I20260812 01:11:42.799422 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.781107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:44147 (local address 127.17.130.62:40367)
0812 01:11:42.784477 (+ 3370us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.784488 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.784538 (+ 50us) server_negotiation.cc:407] Connection header received
0812 01:11:42.784721 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.784729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.784790 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.784957 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:42.786184 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.787696 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.790254 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.791090 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.791230 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.793920 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:42.793924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:42.793925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:42.793936 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:42.796428 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.797421 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.797426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.797427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.797579 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.798205 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.798208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.798208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.798558 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:42.798684 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.799082 (+ 398us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.799230 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3304,"thread_start_us":90,"threads_started":1}
I20260812 01:11:42.800611 8995 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40367
I20260812 01:11:42.800737 8995 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:42.800976 8995 heartbeater.cc:511] Master 127.17.130.62:40367 requested a full tablet report, sending...
I20260812 01:11:42.801869 8684 ts_manager.cc:194] Registered new tserver with Master: 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481)
I20260812 01:11:42.803130 8684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:44147
I20260812 01:11:42.804056 8995 heartbeater.cc:503] Master 127.17.130.62:40367 was elected leader, sending a full tablet report...
I20260812 01:11:42.808502 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.802565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34481 (local address 127.0.0.1:48880)
0812 01:11:42.805293 (+ 2728us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.805326 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.805434 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.805673 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.805676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.805678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:42.805759 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.805761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.806629 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.806630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.807472 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.807475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.808158 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.808160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.808249 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.808262 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.808358 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.808417 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2695,"thread_start_us":109,"threads_started":1}
I20260812 01:11:42.808502 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.802866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.17.130.1:34481)
0812 01:11:42.803064 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.803068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.805410 (+ 2342us) server_negotiation.cc:407] Connection header received
0812 01:11:42.805550 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.805553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.805569 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.805667 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:42.805854 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.806523 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.807570 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.808062 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.808155 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.808219 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.808375 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.808421 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260812 01:11:42.809266 8892 tablet_service.cc:1514] Processing DeleteTablet for tablet 36b61daaf6cf402b942a5efed12f9604 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:42 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:48880
I20260812 01:11:42.809825 9014 tablet_replica.cc:330] stopping tablet replica
I20260812 01:11:42.809962 9014 raft_consensus.cc:2229] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:42.810063 9014 pending_rounds.cc:62] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Trying to abort 1 pending ops.
I20260812 01:11:42.810103 9014 pending_rounds.cc:68] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317492131304116224 op_type: WRITE_OP write_request { tablet_id: "36b61daaf6cf402b942a5efed12f9604" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\372\222\357\323\006\"8\n\ntest-admin\022*\n 11820c70cb0d4e0d90e03b17daebf141\020\001\030\001 \001(\001" signature: "<redacted>""\205\024:\225\320\367\231\302\314H\020\376?\240\025\347\244\"\234*)\316\335\342\263\271\211\222p\202\2744vu\214\204\006\226\353\251VN5\017\210\214\356\323#O\3344\273\236~K_\251w\376\377\201\223\374" signing_key_seq_num: 0 } } request_id { client_id: "56cb79abcd994498803e12249e5c2d5c" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260812 01:11:42.810380 9014 raft_consensus.cc:2258] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:42.810720 9014 ts_tablet_manager.cc:1905] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:42.812590 9014 ts_tablet_manager.cc:1918] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:42.812673 9014 log.cc:1196] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/36b61daaf6cf402b942a5efed12f9604
I20260812 01:11:42.812945 9014 ts_tablet_manager.cc:1939] T 36b61daaf6cf402b942a5efed12f9604 P 3fb37b12da984ee299b7a3a2efade141: Deleting consensus metadata
I20260812 01:11:42.813657 8644 catalog_manager.cc:4841] TS 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481): tablet 36b61daaf6cf402b942a5efed12f9604 (table test-table [id=11820c70cb0d4e0d90e03b17daebf141]) successfully deleted
W20260812 01:11:42.891657 9006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:42.891942 9006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:42.891971 9006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:42.893522 9006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:42.893591 9006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:42.893625 9006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:11:42.895859 9006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:44709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=46777
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:42.896342 9006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:42.896777 9006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:42.903167 9023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.903182 9025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:42.903187 9022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:42.903614 9006 server_base.cc:1034] running on GCE node
I20260812 01:11:42.903913 9006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:42.904232 9006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:42.905445 9006 hybrid_clock.cc:648] HybridClock initialized: now 1786497102905430 us; error 39 us; skew 500 ppm
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:42.908679 9006 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:11:42.909303 9006 webserver.cc:460] Webserver started at http://127.17.130.2:46777/ using document root <none> and password file <none>
I20260812 01:11:42.909576 9006 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:42.909677 9006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:42.911222 9006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:11:42.912101 9036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.912349 9006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:42.912483 9006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "0fe9991259cf44a6981b05c9cac2c08b"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "6e3cea571b38648ede14f93d6f670417"
server_key_iv: "5ebb231dc5fec4aee32243c445099360"
server_key_version: "encryptionkey@0"
I20260812 01:11:42.912904 9006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:42.941897 9006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:42.944696 9006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:42.944904 9006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:42.945205 9006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:42.945571 9006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:42.945641 9006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.945688 9006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:42.945719 9006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:42.952795 9006 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:44709
I20260812 01:11:42.952844 9263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:44709 every 8 connection(s)
I20260812 01:11:42.953303 9006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:11:42.957660 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 9006
I20260812 01:11:42.957969 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:33609
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:40367
--webserver_port=37121
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 01:11:42 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497102, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:42.965981 9270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.954143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:54911)
0812 01:11:42.954364 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:42.954407 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:42.954979 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:42.955272 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:42.955276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:42.956159 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:42.957204 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.957213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.958049 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.958050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.958969 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:42.958972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.959410 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.959411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:42.959553 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.960176 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:42.960185 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:42.962017 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:42.963931 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.963941 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.963945 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.964329 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.964584 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:42.964585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:42.964586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:42.964761 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:42.965032 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:42.965041 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:42.965286 (+ 245us) client_negotiation.cc:769] Sending connection context
0812 01:11:42.965429 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 01:11:42.965652 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260812 01:11:42.966033 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:42.954194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:54911 (local address 127.17.130.62:40367)
0812 01:11:42.954317 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:42.954319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:42.954502 (+ 183us) server_negotiation.cc:407] Connection header received
0812 01:11:42.955097 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:42.955099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:42.955112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:42.955191 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:42.957338 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.957945 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.959075 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:42.959315 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:42.959399 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:42.962161 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:42.962163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:42.962163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:42.962171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:42.963825 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.964449 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.964450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.964450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.964512 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:42.964840 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:42.964840 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:42.964841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:42.964958 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:42.965011 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:42.965590 (+ 579us) server_negotiation.cc:299] Negotiation successful
0812 01:11:42.965863 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 01:11:42.967204 9264 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40367
I20260812 01:11:42.967340 9264 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:42.967589 9264 heartbeater.cc:511] Master 127.17.130.62:40367 requested a full tablet report, sending...
I20260812 01:11:42.968524 8684 ts_manager.cc:194] Registered new tserver with Master: 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709)
I20260812 01:11:42.969187 8684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:54911
W20260812 01:11:43.059015 9271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:43.059275 9271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:43.059316 9271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:43.060520 9271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:43.060592 9271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:43.060639 9271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:11:43.062238 9271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:33609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=37121
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:43.062522 9271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:43.062800 9271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:43.068374 9277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:43.068356 9276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:43.068365 9279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:43.068408 9271 server_base.cc:1034] running on GCE node
I20260812 01:11:43.068778 9271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:43.069120 9271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:43.070300 9271 hybrid_clock.cc:648] HybridClock initialized: now 1786497103070278 us; error 41 us; skew 500 ppm
Aug 12 01:11:43 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497103, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:43.073045 9271 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:11:43.073635 9271 webserver.cc:460] Webserver started at http://127.17.130.3:37121/ using document root <none> and password file <none>
I20260812 01:11:43.073910 9271 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:43.074317 9271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:43.075945 9271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 01:11:43.076715 9291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:43.077034 9271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:43.077190 9271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "af62d748810f4c78b457d64c4facebcb"
format_stamp: "Formatted at 2026-08-12 01:11:42 on dist-test-slave-tfpf"
server_key: "e6dccfa7080f9aca5b08aa136f349707"
server_key_iv: "118b39cc48ec07b87a78e5e2f9efe842"
server_key_version: "encryptionkey@0"
I20260812 01:11:43.077577 9271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:43.090649 9271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:43.093324 9271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:43.093487 9271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:43.093765 9271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:43.094192 9271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:43.094239 9271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:43.094264 9271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:43.094276 9271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:43.101353 9271 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:33609
I20260812 01:11:43.101428 9513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:33609 every 8 connection(s)
I20260812 01:11:43.101791 9271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:11:43.105916 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 9271
Aug 12 01:11:43 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497103, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:11:43.113637 9000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.102617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:52847 (local address 127.17.130.62:40367)
0812 01:11:43.102666 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.102668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.102893 (+ 225us) server_negotiation.cc:407] Connection header received
0812 01:11:43.103424 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.103425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.103437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.103506 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:43.104480 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.104977 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.106112 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.106364 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.106440 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.109295 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:43.109297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:43.109297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:43.109304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:43.111417 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:43.112217 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:43.112218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:43.112218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:43.112275 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:43.112621 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:43.112622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:43.112622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:43.112753 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:43.112809 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.113442 (+ 633us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.113540 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:11:43.113886 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.102614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:52847)
0812 01:11:43.102769 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.102804 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:43.103304 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:43.103562 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:43.103565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:43.104162 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:43.104363 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.104367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.105157 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.105158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.106014 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.106016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.106451 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.106452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.106622 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.107238 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:43.107250 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:43.109183 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:43.111563 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:43.111572 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:43.111575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:43.112114 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:43.112351 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:43.112352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:43.112353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:43.112526 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:43.112829 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:43.112837 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:43.113219 (+ 382us) client_negotiation.cc:769] Sending connection context
0812 01:11:43.113408 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 01:11:43.113608 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260812 01:11:43.114549 9514 heartbeater.cc:347] Connected to a master server at 127.17.130.62:40367
I20260812 01:11:43.114688 9514 heartbeater.cc:464] Registering TS with master...
I20260812 01:11:43.114907 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.106438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.17.130.62:40367)
0812 01:11:43.106586 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.106588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.106666 (+ 78us) server_negotiation.cc:407] Connection header received
0812 01:11:43.106745 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.106745 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.106756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.106833 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:43.107631 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.108368 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.109465 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.109770 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.109840 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.111570 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:43.111571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:43.111572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:43.111581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:43.113569 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:43.114059 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:43.114061 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:43.114061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:43.114142 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:43.114354 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:43.114355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:43.114356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:43.114475 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:43.114663 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.114715 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.114803 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260812 01:11:43.114921 9514 heartbeater.cc:511] Master 127.17.130.62:40367 requested a full tablet report, sending...
I20260812 01:11:43.115504 8684 ts_manager.cc:194] Registered new tserver with Master: af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
I20260812 01:11:43.115926 8684 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:52847
I20260812 01:11:43.115937 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:11:43.120004 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.116357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.17.130.62:40367)
0812 01:11:43.116400 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.116402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.116569 (+ 167us) server_negotiation.cc:407] Connection header received
0812 01:11:43.116632 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.116633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.116695 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.116769 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:11:43.117835 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.118381 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.119238 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.119437 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.119510 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.119634 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:11:43.119729 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:11:43.119791 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.119877 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.119899 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:11:43.120355 8684 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:11:43.122123 8684 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:11:43.124804 8892 tablet_service.cc:1467] Processing CreateTablet for tablet d572adbd755e4d7a99b52af72c5a9784 (DEFAULT_TABLE table=test-table [id=c5af1215e808427c80b7fda4f0ba5756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:43.125145 8892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d572adbd755e4d7a99b52af72c5a9784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:43.126617 8818 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap starting.
I20260812 01:11:43.127019 8818 tablet_bootstrap.cc:654] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:43.127687 8818 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: No bootstrap required, opened a new log
I20260812 01:11:43.127744 8818 ts_tablet_manager.cc:1397] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:43.127843 8818 raft_consensus.cc:348] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.127916 8818 raft_consensus.cc:374] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:43.127965 8818 raft_consensus.cc:729] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb37b12da984ee299b7a3a2efade141, State: Initialized, Role: FOLLOWER
I20260812 01:11:43.128010 8818 consensus_queue.cc:260] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.128093 8818 ts_tablet_manager.cc:1428] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:11:43.129480 9522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.124407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:44709 (local address 127.0.0.1:59524)
0812 01:11:43.124560 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.124571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:43.124629 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:43.125605 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:43.125606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:43.125608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:43.125654 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.125655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.127259 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.127260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.128338 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.128341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.129231 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.129232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.129303 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.129308 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:11:43.129361 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:11:43.129408 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":31,"thread_start_us":63,"threads_started":1}
I20260812 01:11:43.129611 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.124407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:33609 (local address 127.0.0.1:32976)
0812 01:11:43.124475 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.124489 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:43.124541 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:43.125053 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:43.125054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:43.125056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:43.125137 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.125139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.126588 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.126589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.127652 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.127655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.129338 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.129339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.129413 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.129417 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:11:43.129486 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:11:43.129535 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:11:43.129643 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.124647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32976 (local address 127.17.130.3:33609)
0812 01:11:43.124856 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.124860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.124869 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:11:43.124923 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.124925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.124943 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.125247 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:43.125326 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.126456 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.127765 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.129240 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.129337 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.129400 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.129504 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.129547 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260812 01:11:43.129778 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.124859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.17.130.2:44709)
0812 01:11:43.125305 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.125310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.125320 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:11:43.125428 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.125430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.125446 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.125570 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:43.125738 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.127030 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.128432 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.128990 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.129156 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.129235 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.129621 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.129674 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":304,"threads_started":1}
I20260812 01:11:43.130642 9126 tablet_service.cc:1467] Processing CreateTablet for tablet d572adbd755e4d7a99b52af72c5a9784 (DEFAULT_TABLE table=test-table [id=c5af1215e808427c80b7fda4f0ba5756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:43.130733 9382 tablet_service.cc:1467] Processing CreateTablet for tablet d572adbd755e4d7a99b52af72c5a9784 (DEFAULT_TABLE table=test-table [id=c5af1215e808427c80b7fda4f0ba5756]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:11:43.131060 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d572adbd755e4d7a99b52af72c5a9784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:43.131165 9382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d572adbd755e4d7a99b52af72c5a9784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:11:43.134305 9528 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Bootstrap starting.
I20260812 01:11:43.134747 9529 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Bootstrap starting.
I20260812 01:11:43.135519 9528 tablet_bootstrap.cc:654] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:43.135912 9529 tablet_bootstrap.cc:654] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Neither blocks nor log segments found. Creating new log.
I20260812 01:11:43.136363 9528 log.cc:824] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:43.136540 9529 log.cc:824] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:43.137338 9528 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: No bootstrap required, opened a new log
I20260812 01:11:43.137399 9529 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: No bootstrap required, opened a new log
I20260812 01:11:43.137418 9528 ts_tablet_manager.cc:1397] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:11:43.137502 9529 ts_tablet_manager.cc:1397] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:11:43.138931 9528 raft_consensus.cc:348] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.138994 9529 raft_consensus.cc:348] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.139104 9528 raft_consensus.cc:374] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:43.139120 9529 raft_consensus.cc:374] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:11:43.139165 9528 raft_consensus.cc:729] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe9991259cf44a6981b05c9cac2c08b, State: Initialized, Role: FOLLOWER
I20260812 01:11:43.139201 9529 raft_consensus.cc:729] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af62d748810f4c78b457d64c4facebcb, State: Initialized, Role: FOLLOWER
I20260812 01:11:43.139298 9528 consensus_queue.cc:260] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.139371 9529 consensus_queue.cc:260] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.139635 9528 ts_tablet_manager.cc:1428] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:11:43.139678 9264 heartbeater.cc:503] Master 127.17.130.62:40367 was elected leader, sending a full tablet report...
I20260812 01:11:43.139741 9529 ts_tablet_manager.cc:1428] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:11:43.139827 9514 heartbeater.cc:503] Master 127.17.130.62:40367 was elected leader, sending a full tablet report...
W20260812 01:11:43.203991 9269 tablet.cc:2369] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:11:43.285707 8997 tablet.cc:2369] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:11:43.352506 9517 tablet.cc:2369] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:11:43.409256 9008 raft_consensus.cc:482] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:43.409490 9008 raft_consensus.cc:504] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.409900 9008 leader_election.cc:288] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
I20260812 01:11:43.413189 9523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.410286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:38845 (local address 127.17.130.3:33609)
0812 01:11:43.410337 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.410340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.410448 (+ 108us) server_negotiation.cc:407] Connection header received
0812 01:11:43.410529 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.410530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.410538 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.410607 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:43.410745 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.411215 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.412269 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.412682 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.412776 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.412787 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.413036 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.413114 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:11:43.413198 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.410296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:54627 (local address 127.17.130.2:44709)
0812 01:11:43.410345 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.410347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.410382 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:11:43.410491 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.410492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.410512 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.410574 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:11:43.410740 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.411219 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.412312 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.412771 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.412884 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.412893 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.413040 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.413129 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:11:43.413211 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.410143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:44709 (local address 127.17.130.1:54627)
0812 01:11:43.410311 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.410324 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:43.410403 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:43.410596 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:43.410597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:43.410599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:43.410667 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.410668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.411309 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.411311 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.412220 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.412223 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.412896 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.412898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.412963 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.412966 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:11:43.413034 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:11:43.413104 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":52,"thread_start_us":99,"threads_started":1}
I20260812 01:11:43.413220 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.410165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:33609 (local address 127.17.130.1:38845)
0812 01:11:43.410358 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.410369 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:43.410457 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:43.410612 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:43.410613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:43.410616 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:11:43.410675 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.410676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.411302 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.411303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.412171 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:43.412173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.412777 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.412778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:43.412847 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.412850 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:11:43.412913 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 01:11:43.413111 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":39,"thread_start_us":73,"threads_started":1}
I20260812 01:11:43.413659 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d572adbd755e4d7a99b52af72c5a9784" candidate_uuid: "3fb37b12da984ee299b7a3a2efade141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af62d748810f4c78b457d64c4facebcb" is_pre_election: true
I20260812 01:11:43.413828 9414 raft_consensus.cc:2454] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb37b12da984ee299b7a3a2efade141 in term 0.
I20260812 01:11:43.413836 9165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d572adbd755e4d7a99b52af72c5a9784" candidate_uuid: "3fb37b12da984ee299b7a3a2efade141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe9991259cf44a6981b05c9cac2c08b" is_pre_election: true
I20260812 01:11:43.414017 9165 raft_consensus.cc:2454] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fb37b12da984ee299b7a3a2efade141 in term 0.
I20260812 01:11:43.414201 8810 leader_election.cc:302] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb37b12da984ee299b7a3a2efade141, af62d748810f4c78b457d64c4facebcb; no voters:
I20260812 01:11:43.414366 9008 raft_consensus.cc:2781] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:11:43.414461 9008 raft_consensus.cc:482] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:11:43.414503 9008 raft_consensus.cc:3037] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:43.415324 9008 raft_consensus.cc:504] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.415491 9008 leader_election.cc:288] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 1 election: Requested vote from peers 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
I20260812 01:11:43.415738 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d572adbd755e4d7a99b52af72c5a9784" candidate_uuid: "3fb37b12da984ee299b7a3a2efade141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af62d748810f4c78b457d64c4facebcb"
I20260812 01:11:43.415818 9414 raft_consensus.cc:3037] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:43.415791 9165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d572adbd755e4d7a99b52af72c5a9784" candidate_uuid: "3fb37b12da984ee299b7a3a2efade141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe9991259cf44a6981b05c9cac2c08b"
I20260812 01:11:43.415877 9165 raft_consensus.cc:3037] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:11:43.416735 9414 raft_consensus.cc:2454] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb37b12da984ee299b7a3a2efade141 in term 1.
I20260812 01:11:43.416857 9165 raft_consensus.cc:2454] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fb37b12da984ee299b7a3a2efade141 in term 1.
I20260812 01:11:43.416958 8810 leader_election.cc:302] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fb37b12da984ee299b7a3a2efade141, af62d748810f4c78b457d64c4facebcb; no voters:
I20260812 01:11:43.417052 9008 raft_consensus.cc:2781] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:11:43.417223 9008 raft_consensus.cc:686] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 LEADER]: Becoming Leader. State: Replica: 3fb37b12da984ee299b7a3a2efade141, State: Running, Role: LEADER
I20260812 01:11:43.417316 9008 consensus_queue.cc:237] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.418256 8683 catalog_manager.cc:5495] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "3fb37b12da984ee299b7a3a2efade141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } health_report { overall_health: HEALTHY } } }
I20260812 01:11:43.491200 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.487581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.17.130.1:34481)
0812 01:11:43.487761 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:43.487763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:43.487771 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:11:43.487821 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:43.487822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:43.487849 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:43.487974 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:11:43.488113 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.488609 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.489720 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:43.490059 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:43.490182 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:43.490388 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:11:43.490522 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:11:43.490617 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:43.490642 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 01:11:43.490675 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":100,"threads_started":1}
I20260812 01:11:43.493856 9414 raft_consensus.cc:1261] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Refusing update from remote peer 3fb37b12da984ee299b7a3a2efade141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:43.494470 9165 raft_consensus.cc:1261] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Refusing update from remote peer 3fb37b12da984ee299b7a3a2efade141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:11:43.494506 9538 consensus_queue.cc:1035] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:43.494853 9008 consensus_queue.cc:1035] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:11:43.497905 9547 mvcc.cc:205] Tried to move back new op lower bound from 7317492135905026048 to 7317492135597965312. Current Snapshot: MvccSnapshot[applied={T|T < 7317492135905026048}]
I20260812 01:11:43.498571 9549 mvcc.cc:205] Tried to move back new op lower bound from 7317492135905026048 to 7317492135597965312. Current Snapshot: MvccSnapshot[applied={T|T < 7317492135905026048 or (T in {7317492135905026048})}]
I20260812 01:11:43.498853 9550 mvcc.cc:205] Tried to move back new op lower bound from 7317492135905026048 to 7317492135597965312. Current Snapshot: MvccSnapshot[applied={T|T < 7317492135905026048}]
I20260812 01:11:43.500782 8683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:11:43.500916 8683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:11:43.501864 8683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Sending DeleteTablet for 3 replicas of tablet d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:43.502250 9126 tablet_service.cc:1514] Processing DeleteTablet for tablet d572adbd755e4d7a99b52af72c5a9784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:43 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:59524
I20260812 01:11:43.502332 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 8749
I20260812 01:11:43.502322 9382 tablet_service.cc:1514] Processing DeleteTablet for tablet d572adbd755e4d7a99b52af72c5a9784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:11:43 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:32976
I20260812 01:11:43.517330 9564 tablet_replica.cc:330] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: stopping tablet replica
I20260812 01:11:43.517541 9564 raft_consensus.cc:2229] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:43.517696 9564 raft_consensus.cc:2258] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:43.519142 9564 ts_tablet_manager.cc:1905] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:43.521279 9565 tablet_replica.cc:330] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: stopping tablet replica
I20260812 01:11:43.521487 9565 raft_consensus.cc:2229] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:11:43.521524 9564 ts_tablet_manager.cc:1918] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:43.521610 9564 log.cc:1196] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:43.521664 9565 raft_consensus.cc:2258] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:11:43.521910 9564 ts_tablet_manager.cc:1939] T d572adbd755e4d7a99b52af72c5a9784 P 0fe9991259cf44a6981b05c9cac2c08b: Deleting consensus metadata
I20260812 01:11:43.522229 9565 ts_tablet_manager.cc:1905] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:11:43.522449 8645 catalog_manager.cc:4841] TS 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709): tablet d572adbd755e4d7a99b52af72c5a9784 (table test-table [id=c5af1215e808427c80b7fda4f0ba5756]) successfully deleted
I20260812 01:11:43.524084 9565 ts_tablet_manager.cc:1918] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:11:43.524278 9565 log.cc:1196] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:43.524709 9565 ts_tablet_manager.cc:1939] T d572adbd755e4d7a99b52af72c5a9784 P af62d748810f4c78b457d64c4facebcb: Deleting consensus metadata
I20260812 01:11:43.525326 8644 catalog_manager.cc:4841] TS af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609): tablet d572adbd755e4d7a99b52af72c5a9784 (table test-table [id=c5af1215e808427c80b7fda4f0ba5756]) successfully deleted
W20260812 01:11:43.526351 8644 connection.cc:535] client connection to 127.17.130.1:34481 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 01:11:43.526479 8644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:11:43.526530 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 9006
I20260812 01:11:43.533607 9010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.533447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34481 (local address 127.0.0.1:48908)
0812 01:11:43.533509 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.533554 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 01:11:43.533757 8644 catalog_manager.cc:4584] TS 3fb37b12da984ee299b7a3a2efade141 (127.17.130.1:34481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d572adbd755e4d7a99b52af72c5a9784: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34481: connect: Connection refused (error 111)
I20260812 01:11:43.543074 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 9271
I20260812 01:11:43.556142 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 8618
I20260812 01:11:43.574486 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40367
--webserver_interface=127.17.130.62
--webserver_port=44039
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:40367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:43.664014 9572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:43.664258 9572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:43.664283 9572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:43.665510 9572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:11:43.665561 9572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:43.665576 9572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:43.665589 9572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:11:43.665601 9572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:11:43.667237 9572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:40367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:40367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=44039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:43.667537 9572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:43.667831 9572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:43.673233 9584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:43.673233 9581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:43.673278 9582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:43.673480 9572 server_base.cc:1034] running on GCE node
I20260812 01:11:43.674045 9572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:43.674504 9572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:43.675740 9572 hybrid_clock.cc:648] HybridClock initialized: now 1786497103675682 us; error 78 us; skew 500 ppm
Aug 12 01:11:43 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497103, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:43.679287 9572 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:11:43.679877 9572 webserver.cc:460] Webserver started at http://127.17.130.62:44039/ using document root <none> and password file <none>
I20260812 01:11:43.680178 9572 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:43.680231 9572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:43.682184 9572 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:43.683187 9591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:43.683526 9572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:43.683658 9572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "466e199d2f00460c90442b73f005210f"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "ef58460c6f4379d313e98a5b1f632a18"
server_key_iv: "82552d78ce7c4c1dfd1536e3546e361a"
server_key_version: "encryptionkey@0"
I20260812 01:11:43.684104 9572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:43.699836 9572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:43.702435 9572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:43.702572 9572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:43.706868 9572 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:40367
I20260812 01:11:43.706936 9681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:40367 every 8 connection(s)
I20260812 01:11:43.707439 9572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:11:43.709328 9682 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:11:43.710055 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap starting.
I20260812 01:11:43.711203 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 9572
I20260812 01:11:43.711505 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:34481
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:40367
--webserver_port=45829
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:43.720479 9682 log.cc:824] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:43.722849 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:11:43.723191 9682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f: Bootstrap complete.
I20260812 01:11:43.725728 9682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:43.725996 9682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466e199d2f00460c90442b73f005210f, State: Initialized, Role: FOLLOWER
I20260812 01:11:43.726202 9682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:43.726285 9682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:11:43.726364 9682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:11:43.726428 9682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 2 FOLLOWER]: Advancing to term 3
I20260812 01:11:43.727626 9682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:43.727759 9682 leader_election.cc:302] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 466e199d2f00460c90442b73f005210f; no voters:
I20260812 01:11:43.727953 9682 leader_election.cc:288] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 01:11:43.727993 9688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 3 FOLLOWER]: Leader election won for term 3
I20260812 01:11:43.728106 9688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [term 3 LEADER]: Becoming Leader. State: Replica: 466e199d2f00460c90442b73f005210f, State: Running, Role: LEADER
I20260812 01:11:43.728199 9688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } }
I20260812 01:11:43.728262 9682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:11:43.728494 9692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "466e199d2f00460c90442b73f005210f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } } }
I20260812 01:11:43.728778 9692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:43.728698 9694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 466e199d2f00460c90442b73f005210f. Latest consensus state: current_term: 3 leader_uuid: "466e199d2f00460c90442b73f005210f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "466e199d2f00460c90442b73f005210f" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 40367 } } }
I20260812 01:11:43.729198 9707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:11:43.729287 9694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 466e199d2f00460c90442b73f005210f [sys.catalog]: This master's current role is: LEADER
I20260812 01:11:43.730221 9707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:11:43.730352 9707 catalog_manager.cc:1259] Loaded cluster ID: 5163b671954747a2a6b40e34730ad003
I20260812 01:11:43.730407 9707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 01:11:43.731093 9707 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1c375279ad google::LogMessage::Fail() at ??:0
@ 0x7f1c3752a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1c375274de google::LogMessage::Flush() at ??:0
@ 0x7f1c37528a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1c3f4480f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f1c3f4518b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1c393f50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1c393ea954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1c3a24e6db start_thread at ??:0
@ 0x7f1c3602561f clone at ??:0
W20260812 01:11:43.826910 9684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:43.827145 9684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:43.827179 9684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:43.829164 9684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:43.829236 9684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:43.829272 9684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:11:43.831357 9684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:34481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=45829
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:43.831687 9684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:43.832176 9684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:43.840509 9749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:43.840657 9754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:43.840547 9751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:43.840922 9684 server_base.cc:1034] running on GCE node
I20260812 01:11:43.841219 9684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:43.841590 9684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:43.842826 9684 hybrid_clock.cc:648] HybridClock initialized: now 1786497103842797 us; error 42 us; skew 500 ppm
Aug 12 01:11:43 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497103, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:43.845920 9684 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:11:43.846541 9684 webserver.cc:460] Webserver started at http://127.17.130.1:45829/ using document root <none> and password file <none>
I20260812 01:11:43.846848 9684 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:43.846902 9684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:43.848426 9684 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:11:43.849669 9765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:43.850036 9684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:11:43.850195 9684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "3fb37b12da984ee299b7a3a2efade141"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "1db89fe49b6a03d3ccfa79d755717faa"
server_key_iv: "1da4fab3584fe4d50c8e77275793df0b"
server_key_version: "encryptionkey@0"
I20260812 01:11:43.850767 9684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:43.899766 9684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:43.902568 9684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:43.902706 9684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:43.903045 9684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:43.903575 9781 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:11:43.906366 9684 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:11:43.906471 9684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 01:11:43.906526 9684 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:11:43.907313 9684 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:11:43.907375 9684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 01:11:43.907454 9781 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap starting.
I20260812 01:11:43.916438 9684 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:34481
I20260812 01:11:43.916949 9684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:11:43.917428 9955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:34481 every 8 connection(s)
I20260812 01:11:43.919754 9781 log.cc:824] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Log is configured to *not* fsync() on all Append() calls
I20260812 01:11:43.920255 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 9684
I20260812 01:11:43.920557 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:44709
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:40367
--webserver_port=46777
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:11:43.923297 9781 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:11:43.923660 9781 tablet_bootstrap.cc:492] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Bootstrap complete.
I20260812 01:11:43.924983 9781 ts_tablet_manager.cc:1397] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.009s
I20260812 01:11:43.926671 9781 raft_consensus.cc:348] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.927151 9781 raft_consensus.cc:729] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fb37b12da984ee299b7a3a2efade141, State: Initialized, Role: FOLLOWER
I20260812 01:11:43.927343 9781 consensus_queue.cc:260] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:43.927651 9781 ts_tablet_manager.cc:1428] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260812 01:11:43.981565 9963 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:11:43.917814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:46005)
0812 01:11:43.918004 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.918061 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:43.918636 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:43.981325 (+ 62689us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
W20260812 01:11:43.981938 9959 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:40367 (0 consecutive failures): Network error: Failed to ping master at 127.17.130.62:40367: Client connection negotiation failed: client connection to 127.17.130.62:40367: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:11:43.982339 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.982201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:38145)
0812 01:11:43.982258 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.982311 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:11:43.982803 9963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:43.982699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:55695)
0812 01:11:43.982752 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:43.982782 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:11:43.983021 9959 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:11:44.022552 9966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:44.022804 9966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:44.022835 9966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:44.024154 9966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:44.024216 9966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:44.024250 9966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:11:44.026041 9966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:44709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=46777
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:44.026383 9966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:44.026630 9966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:44.033049 9978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:44.033161 9983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:44.033423 9981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:44.033473 9966 server_base.cc:1034] running on GCE node
I20260812 01:11:44.033763 9966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:44.034003 9966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:44.035194 9966 hybrid_clock.cc:648] HybridClock initialized: now 1786497104035172 us; error 57 us; skew 500 ppm
Aug 12 01:11:44 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497104, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:44.038513 9966 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:11:44.039106 9966 webserver.cc:460] Webserver started at http://127.17.130.2:46777/ using document root <none> and password file <none>
I20260812 01:11:44.039407 9966 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:44.039491 9966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:44.041110 9966 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:44.041837 9993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:44.042090 9966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:44.042217 9966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "0fe9991259cf44a6981b05c9cac2c08b"
format_stamp: "Formatted at 2026-08-12 01:11:41 on dist-test-slave-tfpf"
server_key: "6e3cea571b38648ede14f93d6f670417"
server_key_iv: "5ebb231dc5fec4aee32243c445099360"
server_key_version: "encryptionkey@0"
I20260812 01:11:44.042639 9966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:44.060956 9966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:44.063686 9966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:44.063899 9966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:44.064213 9966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:44.064595 9966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:44.064661 9966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:44.064710 9966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:44.064765 9966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:44.072491 9966 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:44709
I20260812 01:11:44.072541 10195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:44709 every 8 connection(s)
I20260812 01:11:44.072963 9966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:11:44.074293 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.073809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:32975)
0812 01:11:44.074042 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.074141 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
W20260812 01:11:44.074949 10198 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:40367 (0 consecutive failures): Network error: Failed to ping master at 127.17.130.62:40367: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
I20260812 01:11:44.075491 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.075365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:60715)
0812 01:11:44.075429 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.075465 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 01:11:44.075901 10204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.075803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:46673)
0812 01:11:44.075851 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.075883 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 01:11:44.076015 10198 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:11:44.080294 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 9966
I20260812 01:11:44.080595 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:33609
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:40367
--webserver_port=37121
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:11:44.168992 10205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:11:44.169386 10205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:11:44.169431 10205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:11:44.170632 10205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:11:44.170702 10205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:11:44.170753 10205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:11:44.172324 10205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:33609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=37121
--tserver_master_addrs=127.17.130.62:40367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:11:44.172642 10205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:11:44.172904 10205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:11:44.178647 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:44.178898 10221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:11:44.178951 10217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:11:44.178982 10205 server_base.cc:1034] running on GCE node
I20260812 01:11:44.179225 10205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:11:44.179534 10205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:11:44.180744 10205 hybrid_clock.cc:648] HybridClock initialized: now 1786497104180722 us; error 32 us; skew 500 ppm
Aug 12 01:11:44 dist-test-slave-tfpf krb5kdc[7650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497104, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:11:44.184006 10205 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:11:44.184659 10205 webserver.cc:460] Webserver started at http://127.17.130.3:37121/ using document root <none> and password file <none>
I20260812 01:11:44.184963 10205 fs_manager.cc:359] Metadata directory not provided
I20260812 01:11:44.185042 10205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:11:44.187256 10205 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:44.188236 10234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:44.188484 10205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:11:44.188635 10205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "af62d748810f4c78b457d64c4facebcb"
format_stamp: "Formatted at 2026-08-12 01:11:42 on dist-test-slave-tfpf"
server_key: "e6dccfa7080f9aca5b08aa136f349707"
server_key_iv: "118b39cc48ec07b87a78e5e2f9efe842"
server_key_version: "encryptionkey@0"
I20260812 01:11:44.189288 10205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:11:44.222612 10205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:11:44.225293 10205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:11:44.225478 10205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:11:44.225752 10205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:11:44.226100 10205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:11:44.226163 10205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:44.226208 10205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:11:44.226236 10205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:11:44.234990 10205 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:33609
I20260812 01:11:44.235080 10427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:33609 every 8 connection(s)
I20260812 01:11:44.235627 10205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:11:44.236893 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.236450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:49705)
0812 01:11:44.236637 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.236746 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
W20260812 01:11:44.237541 10431 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:40367 (0 consecutive failures): Network error: Failed to ping master at 127.17.130.62:40367: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
I20260812 01:11:44.237882 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.237772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:44583)
0812 01:11:44.237825 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.237857 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:11:44.238356 10439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.238216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:42775)
0812 01:11:44.238300 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.238333 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 01:11:44.238533 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 10205
W20260812 01:11:44.238570 10431 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:11:44.983717 10443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:44.983450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:55571)
0812 01:11:44.983605 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:44.983649 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260812 01:11:45.076654 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.076422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:49451)
0812 01:11:45.076563 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:45.076599 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260812 01:11:45.239269 10446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.238998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:58433)
0812 01:11:45.239166 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:45.239205 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260812 01:11:45.509024 10447 raft_consensus.cc:482] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:45.509230 10447 raft_consensus.cc:504] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:45.509640 10447 leader_election.cc:288] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
Aug 12 01:11:45 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497103, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.3@KRBTEST.COM
Aug 12 01:11:45 dist-test-slave-tfpf krb5kdc[7650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497103, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.2@KRBTEST.COM
I20260812 01:11:45.529241 10454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.510003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:42975 (local address 127.17.130.3:33609)
0812 01:11:45.510171 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:45.510184 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:45.510223 (+ 39us) server_negotiation.cc:407] Connection header received
0812 01:11:45.510834 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:45.510837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:45.510895 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:45.511071 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:45.512198 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.513669 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.515276 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.515910 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.516040 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:45.524232 (+ 8192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:45.524240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:45.524241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:45.524259 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:45.526245 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:45.526842 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:45.526847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:45.526848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:45.527023 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:45.527532 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:45.527536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:45.527536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:45.528108 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:45.528212 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:45.528905 (+ 693us) server_negotiation.cc:299] Negotiation successful
0812 01:11:45.529115 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
I20260812 01:11:45.529395 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.509880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:33609 (local address 127.17.130.1:42975)
0812 01:11:45.510034 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:45.510046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:45.510107 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:45.511165 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:45.511166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:45.511752 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:45.512036 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:45.512044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.513807 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.513808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:45.515166 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:45.515168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.516178 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.516179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:45.516224 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:45.521668 (+ 5444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:45.521686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:45.524042 (+ 2356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:45.526357 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:45.526369 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:45.526372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:45.526750 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:45.527177 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:45.527178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:45.527178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:45.527314 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:45.528252 (+ 938us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:45.528270 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:45.528503 (+ 233us) client_negotiation.cc:769] Sending connection context
0812 01:11:45.528673 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 01:11:45.529231 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":57,"thread_start_us":89,"threads_started":1}
I20260812 01:11:45.529611 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.509880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:44709 (local address 127.17.130.1:45119)
0812 01:11:45.510034 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:45.510046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:11:45.510107 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:11:45.511160 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:11:45.511164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:11:45.511752 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:11:45.512035 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:45.512044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.513798 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.513799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:45.515150 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:11:45.515154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.516016 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.516017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:11:45.516167 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:45.521668 (+ 5501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:11:45.521686 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:11:45.524235 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:11:45.526496 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:45.526498 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:45.526498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:45.526759 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:45.527397 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:11:45.527397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:11:45.527397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:11:45.527443 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:11:45.529165 (+ 1722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:11:45.529167 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:11:45.529230 (+ 63us) client_negotiation.cc:769] Sending connection context
0812 01:11:45.529324 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 01:11:45.529485 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":99,"thread_start_us":78,"threads_started":1}
I20260812 01:11:45.530081 10457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.510110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:45119 (local address 127.17.130.2:44709)
0812 01:11:45.510268 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 01:11:45.510274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:11:45.510299 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:11:45.510834 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:11:45.510837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:11:45.510895 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:11:45.511068 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:11:45.512198 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.513669 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.515276 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:11:45.515910 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:11:45.516040 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:11:45.524405 (+ 8365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:11:45.524410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:11:45.524410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:11:45.524422 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:11:45.526390 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:45.526900 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:45.526904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:45.526904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:45.527023 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:11:45.527551 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:11:45.527553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:11:45.527553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:11:45.528108 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:11:45.528212 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 01:11:45.529827 (+ 1615us) server_negotiation.cc:299] Negotiation successful
0812 01:11:45.529968 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
W20260812 01:11:45.530616 9769 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
W20260812 01:11:45.531105 9771 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:45.531195 9771 leader_election.cc:302] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb37b12da984ee299b7a3a2efade141; no voters: 0fe9991259cf44a6981b05c9cac2c08b, af62d748810f4c78b457d64c4facebcb
I20260812 01:11:45.531343 10447 raft_consensus.cc:2726] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:11:45.984485 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:45.984310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:41385)
0812 01:11:45.984395 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:45.984438 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 01:11:46.077611 10459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:46.077329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:41975)
0812 01:11:46.077491 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:46.077539 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260812 01:11:46.240223 10460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:46.239939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:39737)
0812 01:11:46.240127 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:46.240173 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":75,"threads_started":1}
I20260812 01:11:46.985447 10461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:46.985194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:33757)
0812 01:11:46.985343 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:46.985384 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260812 01:11:47.078534 10462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:47.078256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:39457)
0812 01:11:47.078432 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:47.078476 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260812 01:11:47.241250 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:47.240881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:45571)
0812 01:11:47.241110 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:47.241169 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":92,"threads_started":1}
I20260812 01:11:47.256695 10470 raft_consensus.cc:482] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:47.256834 10470 raft_consensus.cc:504] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:47.257130 10470 leader_election.cc:288] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
W20260812 01:11:47.257867 9771 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
W20260812 01:11:47.257973 9769 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:47.258116 9769 leader_election.cc:302] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb37b12da984ee299b7a3a2efade141; no voters: 0fe9991259cf44a6981b05c9cac2c08b, af62d748810f4c78b457d64c4facebcb
I20260812 01:11:47.258249 10470 raft_consensus.cc:2726] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:11:47.986284 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:47.986060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:49909)
0812 01:11:47.986196 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:47.986234 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260812 01:11:48.079525 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:48.079233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:46963)
0812 01:11:48.079418 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:48.079469 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260812 01:11:48.242118 10474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:48.241841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:48827)
0812 01:11:48.242022 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:48.242068 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260812 01:11:48.987304 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:48.986976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:38841)
0812 01:11:48.987202 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:48.987251 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
I20260812 01:11:49.080492 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:49.080221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:59565)
0812 01:11:49.080379 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:49.080416 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260812 01:11:49.242969 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:49.242733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:56887)
0812 01:11:49.242872 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:49.242906 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260812 01:11:49.988227 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:49.987972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:49819)
0812 01:11:49.988136 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:49.988178 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260812 01:11:50.081434 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:50.081053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:55727)
0812 01:11:50.081318 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:50.081382 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":105,"threads_started":1}
I20260812 01:11:50.243774 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:50.243524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:52417)
0812 01:11:50.243684 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:50.243717 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260812 01:11:50.279657 10495 raft_consensus.cc:482] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:50.279810 10495 raft_consensus.cc:504] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:50.280184 10495 leader_election.cc:288] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
W20260812 01:11:50.280354 9769 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
W20260812 01:11:50.280403 9771 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:50.280428 9769 leader_election.cc:302] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb37b12da984ee299b7a3a2efade141; no voters: 0fe9991259cf44a6981b05c9cac2c08b, af62d748810f4c78b457d64c4facebcb
I20260812 01:11:50.280560 10495 raft_consensus.cc:2726] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:11:50.989248 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:50.988888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:45371)
0812 01:11:50.989128 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:50.989192 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260812 01:11:51.082402 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:51.082067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:53663)
0812 01:11:51.082308 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:51.082352 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
I20260812 01:11:51.244702 10500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:51.244413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:33263)
0812 01:11:51.244593 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:51.244641 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":77,"threads_started":1}
I20260812 01:11:51.990173 10502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:51.989899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:56185)
0812 01:11:51.990068 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:51.990116 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
I20260812 01:11:52.083632 10503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:52.083220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:59241)
0812 01:11:52.083498 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:52.083571 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":153,"threads_started":1}
I20260812 01:11:52.245666 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:52.245394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:41943)
0812 01:11:52.245565 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:52.245613 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260812 01:11:52.991389 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:52.991122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:40921)
0812 01:11:52.991289 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:52.991336 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260812 01:11:53.084678 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:53.084353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:45917)
0812 01:11:53.084576 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:53.084624 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":76,"threads_started":1}
I20260812 01:11:53.246693 10512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:53.246322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:57191)
0812 01:11:53.246577 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:53.246631 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":135,"threads_started":1}
I20260812 01:11:53.992331 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:53.992036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:45499)
0812 01:11:53.992219 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:53.992277 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260812 01:11:54.037858 10517 raft_consensus.cc:482] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:11:54.037998 10517 raft_consensus.cc:504] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fe9991259cf44a6981b05c9cac2c08b" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 44709 } } peers { permanent_uuid: "af62d748810f4c78b457d64c4facebcb" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 33609 } } peers { permanent_uuid: "3fb37b12da984ee299b7a3a2efade141" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34481 } }
I20260812 01:11:54.038226 10517 leader_election.cc:288] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709), af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609)
W20260812 01:11:54.038959 9771 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fe9991259cf44a6981b05c9cac2c08b (127.17.130.2:44709): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
W20260812 01:11:54.039047 9769 leader_election.cc:341] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer af62d748810f4c78b457d64c4facebcb (127.17.130.3:33609): Not found: Tablet not found: d572adbd755e4d7a99b52af72c5a9784
I20260812 01:11:54.039129 9769 leader_election.cc:302] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3fb37b12da984ee299b7a3a2efade141; no voters: 0fe9991259cf44a6981b05c9cac2c08b, af62d748810f4c78b457d64c4facebcb
I20260812 01:11:54.039216 10517 raft_consensus.cc:2726] T d572adbd755e4d7a99b52af72c5a9784 P 3fb37b12da984ee299b7a3a2efade141 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:11:54.085857 10518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:54.085584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:36169)
0812 01:11:54.085751 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:54.085801 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
I20260812 01:11:54.247597 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:54.247332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:42183)
0812 01:11:54.247496 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:54.247542 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260812 01:11:54.993324 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:54.992960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:55749)
0812 01:11:54.993220 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:54.993272 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":214,"threads_started":1}
I20260812 01:11:55.086735 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:55.086436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:34925)
0812 01:11:55.086637 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:55.086681 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":105,"threads_started":1}
I20260812 01:11:55.248546 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:55.248244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:46043)
0812 01:11:55.248435 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:55.248475 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":99,"threads_started":1}
I20260812 01:11:55.994158 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:55.993829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:33287)
0812 01:11:55.994037 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:55.994089 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":111,"threads_started":1}
I20260812 01:11:56.087692 10524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:56.087394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:36349)
0812 01:11:56.087587 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:56.087636 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260812 01:11:56.249783 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:56.249471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:49451)
0812 01:11:56.249685 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:56.249732 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
I20260812 01:11:56.995177 10528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:56.994751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:52541)
0812 01:11:56.994966 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:56.995109 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260812 01:11:57.088610 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:57.088290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:49101)
0812 01:11:57.088502 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:57.088558 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
I20260812 01:11:57.250739 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:57.250457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:50165)
0812 01:11:57.250643 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:57.250688 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260812 01:11:57.996003 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:57.995745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:43923)
0812 01:11:57.995907 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:57.995948 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260812 01:11:58.089529 10537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:58.089239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:47467)
0812 01:11:58.089420 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:58.089477 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260812 01:11:58.251888 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:58.251449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:55353)
0812 01:11:58.251796 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:58.251835 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":275,"threads_started":1}
I20260812 01:11:58.996953 10545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:58.996643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.1:44029)
0812 01:11:58.996849 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:58.996903 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":124,"threads_started":1}
I20260812 01:11:59.090862 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:59.090444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.2:34527)
0812 01:11:59.090643 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:59.090702 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
I20260812 01:11:59.242364 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 9684
I20260812 01:11:59.252840 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:11:59.252614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:40367 (local address 127.17.130.3:43437)
0812 01:11:59.252746 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 01:11:59.252788 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:40367: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":78,"threads_started":1}
I20260812 01:11:59.255800 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 9966
I20260812 01:11:59.277379 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 10205
2026-08-12T01:11:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19313 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:11:59 dist-test-slave-tfpf krb5kdc[10551](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:11:59 dist-test-slave-tfpf krb5kdc[10551](info): set up 2 sockets
Aug 12 01:11:59 dist-test-slave-tfpf krb5kdc[10551](info): commencing operation
krb5kdc: starting...
W20260812 01:12:00.651479 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:12:00 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:12:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:12:00.803884 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46017
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:00.907591 10567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:00.907852 10567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:00.907879 10567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:00.909921 10567 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 01:12:00.909996 10567 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 01:12:00.910009 10567 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 01:12:00.910022 10567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:12:00.910036 10567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:00.910049 10567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:00.910063 10567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:12:00.910127 10567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:12:00.911957 10567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:00.912292 10567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:00.912572 10567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:00.920367 10577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:00.920367 10580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:00.920744 10567 server_base.cc:1034] running on GCE node
W20260812 01:12:00.920640 10578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:00.921259 10567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:00.921595 10567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:00.922802 10567 hybrid_clock.cc:648] HybridClock initialized: now 1786497120922767 us; error 63 us; skew 500 ppm
Aug 12 01:12:00 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497120, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:00.925765 10567 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:12:00.926397 10567 webserver.cc:460] Webserver started at http://127.17.130.62:44773/ using document root <none> and password file <none>
I20260812 01:12:00.926681 10567 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:00.926757 10567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:00.926921 10567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:12:00.928539 10567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "ea233c0ef0714d78b89f5d11f3cd388e"
format_stamp: "Formatted at 2026-08-12 01:12:00 on dist-test-slave-tfpf"
server_key: "d4a53edaf0463d43590c6032fd24ecaf"
server_key_iv: "4745393662fe9d9d5b2b29088382b2fd"
server_key_version: "encryptionkey@0"
I20260812 01:12:00.928979 10567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "ea233c0ef0714d78b89f5d11f3cd388e"
format_stamp: "Formatted at 2026-08-12 01:12:00 on dist-test-slave-tfpf"
server_key: "d4a53edaf0463d43590c6032fd24ecaf"
server_key_iv: "4745393662fe9d9d5b2b29088382b2fd"
server_key_version: "encryptionkey@0"
I20260812 01:12:00.930857 10567 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 01:12:00.931851 10596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:00.932678 10567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:00.933183 10567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "ea233c0ef0714d78b89f5d11f3cd388e"
format_stamp: "Formatted at 2026-08-12 01:12:00 on dist-test-slave-tfpf"
server_key: "d4a53edaf0463d43590c6032fd24ecaf"
server_key_iv: "4745393662fe9d9d5b2b29088382b2fd"
server_key_version: "encryptionkey@0"
I20260812 01:12:00.933322 10567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:00.950634 10567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:00.953326 10567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:00.953514 10567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:00.959504 10567 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46017
I20260812 01:12:00.959611 10683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46017 every 8 connection(s)
I20260812 01:12:00.960031 10567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:12:00.961127 10685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:00.961413 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 10567
I20260812 01:12:00.961530 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:12:00.961750 17928 external_mini_cluster.cc:1428] Setting key fe8f14f0da6c176973264a18d70ec685
I20260812 01:12:00.964592 10685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap starting.
I20260812 01:12:00.965552 10685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:00.966017 10685 log.cc:824] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:00.967693 10685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: No bootstrap required, opened a new log
Aug 12 01:12:00 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:12:00.971448 10685 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:00.971724 10685 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:00.971789 10685 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea233c0ef0714d78b89f5d11f3cd388e, State: Initialized, Role: FOLLOWER
I20260812 01:12:00.971958 10685 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:00.972064 10685 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:12:00.972107 10685 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:12:00.972201 10685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:00.973719 10685 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:00.973896 10685 leader_election.cc:302] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea233c0ef0714d78b89f5d11f3cd388e; no voters:
I20260812 01:12:00.974164 10685 leader_election.cc:288] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:12:00.974247 10695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:12:00.974458 10695 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 LEADER]: Becoming Leader. State: Replica: ea233c0ef0714d78b89f5d11f3cd388e, State: Running, Role: LEADER
I20260812 01:12:00.974563 10685 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:12:00.974570 10695 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:00.974989 10695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } } }
I20260812 01:12:00.975075 10695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: This master's current role is: LEADER
I20260812 01:12:00.975230 10695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea233c0ef0714d78b89f5d11f3cd388e. Latest consensus state: current_term: 1 leader_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } } }
I20260812 01:12:00.975296 10695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: This master's current role is: LEADER
I20260812 01:12:00.975442 10704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:12:00.976058 10704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:12:00.978128 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:00.962618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.17.130.62:46017)
0812 01:12:00.962832 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:00.962841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:00.962869 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:12:00.963298 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:00.963305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:00.963351 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:00.963482 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:00.964370 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:00.965510 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:00.966431 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:00.966860 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:00.966990 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:00.970573 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:00.970580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:00.970580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:00.970601 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:00.973301 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:00.973975 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:00.973979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:00.973980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:00.974120 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:00.974384 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:00.974385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:00.974385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:00.974764 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:00.975731 (+ 967us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:00.976062 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 01:12:00.977912 (+ 1850us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260812 01:12:00.978132 10704 catalog_manager.cc:1347] Generated new cluster ID: 346d8fcd9eb94117b46ffbdeac7c6972
I20260812 01:12:00.978380 10704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:12:00.987401 10704 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:12:00.987977 10704 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:12:01.011466 10704 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Generated new TSK 0
I20260812 01:12:01.011727 10704 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:12:01.049399 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46017
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:01.166008 10734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:01.166247 10734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:01.166273 10734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:01.167364 10734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:01.167402 10734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:01.167429 10734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:12:01.168849 10734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:01.169138 10734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:01.169378 10734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:01.175518 10741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:01.175516 10744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:01.175545 10742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:01.176036 10734 server_base.cc:1034] running on GCE node
I20260812 01:12:01.176295 10734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:01.176553 10734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:01.177722 10734 hybrid_clock.cc:648] HybridClock initialized: now 1786497121177706 us; error 31 us; skew 500 ppm
Aug 12 01:12:01 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497121, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:01.180709 10734 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:12:01.181366 10734 webserver.cc:460] Webserver started at http://127.17.130.1:39333/ using document root <none> and password file <none>
I20260812 01:12:01.181627 10734 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:01.181701 10734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:01.181864 10734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:12:01.183019 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "9f4b2028807946299b80049c8b32d875"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "08fa3114af6f031cfa79c6ac27b2c88e"
server_key_iv: "c4371e9b30762725742467c873d8f17b"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.183411 10734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "9f4b2028807946299b80049c8b32d875"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "08fa3114af6f031cfa79c6ac27b2c88e"
server_key_iv: "c4371e9b30762725742467c873d8f17b"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.185371 10734 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:12:01.186547 10756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.187006 10734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:12:01.187165 10734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "9f4b2028807946299b80049c8b32d875"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "08fa3114af6f031cfa79c6ac27b2c88e"
server_key_iv: "c4371e9b30762725742467c873d8f17b"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.187271 10734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:01.216827 10734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:01.219678 10734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:01.219908 10734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:01.220230 10734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:01.220593 10734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:01.220654 10734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.220700 10734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:01.220743 10734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.228097 10734 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:40657
I20260812 01:12:01.228168 10947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:40657 every 8 connection(s)
I20260812 01:12:01.228533 10734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:12:01.229544 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 10734
I20260812 01:12:01.229678 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:12:01.230046 17928 external_mini_cluster.cc:1428] Setting key 22d01b3e85452936d053ec860d98e2a4
Aug 12 01:12:01 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497121, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
I20260812 01:12:01.242551 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.229555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:34207 (local address 127.17.130.62:46017)
0812 01:12:01.229615 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.229617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.229929 (+ 312us) server_negotiation.cc:407] Connection header received
0812 01:12:01.230659 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.230660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.230672 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.230741 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:01.232207 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.232746 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.234784 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.235101 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.235213 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.238420 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:01.238422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:01.238422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:01.238434 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:01.240498 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.241118 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.241120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.241120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.241196 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.241535 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.241536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.241537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.241720 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:01.241812 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.242178 (+ 366us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.242411 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:12:01.242583 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.229365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:34207)
0812 01:12:01.229742 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.229805 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.230549 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.230804 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.230808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.231726 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:01.232032 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.232040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.232867 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.232869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.234592 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.234602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.235281 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.235283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.235425 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.236155 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:01.236165 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:01.238250 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:01.240642 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:01.240656 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:01.240659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:01.240967 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:01.241311 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:01.241312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:01.241313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:01.241447 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:01.241838 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:01.241847 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:01.242009 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.242117 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.242278 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":111,"threads_started":1}
I20260812 01:12:01.243340 10950 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46017
I20260812 01:12:01.243495 10950 heartbeater.cc:464] Registering TS with master...
I20260812 01:12:01.243741 10950 heartbeater.cc:511] Master 127.17.130.62:46017 requested a full tablet report, sending...
I20260812 01:12:01.244609 10628 ts_manager.cc:194] Registered new tserver with Master: 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657)
I20260812 01:12:01.245502 10628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:34207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:12:01.268280 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46017
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:01.362397 10956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:01.362635 10956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:01.362670 10956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:01.364135 10956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:01.364202 10956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:01.364230 10956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:12:01.366178 10956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:01.366478 10956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:01.366796 10956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:01.372889 10965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:01.372895 10962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:01.372891 10963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:01.373220 10956 server_base.cc:1034] running on GCE node
I20260812 01:12:01.373446 10956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:01.373801 10956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:01.375063 10956 hybrid_clock.cc:648] HybridClock initialized: now 1786497121374964 us; error 111 us; skew 500 ppm
Aug 12 01:12:01 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497121, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:01.378945 10956 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:12:01.379503 10956 webserver.cc:460] Webserver started at http://127.17.130.2:42247/ using document root <none> and password file <none>
I20260812 01:12:01.379809 10956 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:01.379890 10956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:01.380069 10956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:12:01.381382 10956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "9d1a5fe6e8dd4f03d2b738b782f2cdc1"
server_key_iv: "74c01ba0060f0cc28fbd861fecdc1c4b"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.381810 10956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "9d1a5fe6e8dd4f03d2b738b782f2cdc1"
server_key_iv: "74c01ba0060f0cc28fbd861fecdc1c4b"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.383658 10956 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 01:12:01.384708 10978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.385205 10956 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:01.385447 10956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "9d1a5fe6e8dd4f03d2b738b782f2cdc1"
server_key_iv: "74c01ba0060f0cc28fbd861fecdc1c4b"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.385589 10956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:01.416039 10956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:01.419128 10956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:01.419320 10956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:01.419718 10956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:01.420130 10956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:01.420197 10956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.420244 10956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:01.420274 10956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.426913 10956 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:39631
I20260812 01:12:01.427050 11184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:39631 every 8 connection(s)
I20260812 01:12:01.427380 10956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:12:01 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497121, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:12:01.436332 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 10956
I20260812 01:12:01.436480 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:12:01.436754 17928 external_mini_cluster.cc:1428] Setting key b73075ccc2f76529f89d129da8d8e7eb
I20260812 01:12:01.439220 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.428259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:42471 (local address 127.17.130.62:46017)
0812 01:12:01.428355 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.428358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.428562 (+ 204us) server_negotiation.cc:407] Connection header received
0812 01:12:01.429153 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.429155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.429176 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.429241 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:01.430730 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.431317 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.432410 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.432620 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.432691 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.435570 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:01.435571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:01.435572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:01.435580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:01.437683 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.438179 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.438181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.438181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.438248 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.438499 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.438500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.438500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.438637 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:01.438721 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.439027 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.439127 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 01:12:01.439442 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.428174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:42471)
0812 01:12:01.428341 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.428400 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.428917 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.429464 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.429471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.430354 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:01.430585 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.430593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.431437 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.431438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.432301 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.432303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.432818 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.432820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.433025 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.433814 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:01.433824 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:01.435443 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:01.437783 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:01.437794 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:01.437797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:01.438078 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:01.438303 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:01.438304 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:01.438304 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:01.438426 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:01.438712 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:01.438720 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:01.438874 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.438990 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.439174 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260812 01:12:01.440052 11186 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46017
I20260812 01:12:01.440187 11186 heartbeater.cc:464] Registering TS with master...
I20260812 01:12:01.440392 11186 heartbeater.cc:511] Master 127.17.130.62:46017 requested a full tablet report, sending...
I20260812 01:12:01.440838 10628 ts_manager.cc:194] Registered new tserver with Master: 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
I20260812 01:12:01.441296 10628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:42471
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:12:01.469913 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46017
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:01.576432 11194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:01.576665 11194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:01.576709 11194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:01.577982 11194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:01.578040 11194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:01.578070 11194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:12:01.579591 11194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:01.579823 11194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:01.580101 11194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:01.586040 11203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:01.586045 11202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:01.586045 11207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:01.586490 11194 server_base.cc:1034] running on GCE node
I20260812 01:12:01.586745 11194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:01.587052 11194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:01.588224 11194 hybrid_clock.cc:648] HybridClock initialized: now 1786497121588201 us; error 41 us; skew 500 ppm
Aug 12 01:12:01 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497121, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:01.591822 11194 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:12:01.592398 11194 webserver.cc:460] Webserver started at http://127.17.130.3:42237/ using document root <none> and password file <none>
I20260812 01:12:01.592785 11194 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:01.592854 11194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:01.593025 11194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:12:01.594280 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "2e94f241a5fb4612865995df1bc22299"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "42d988199f4034ecafd5fa1e6b4a67f4"
server_key_iv: "1d6ca318cd61bde73ad671163e7a3130"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.594688 11194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "2e94f241a5fb4612865995df1bc22299"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "42d988199f4034ecafd5fa1e6b4a67f4"
server_key_iv: "1d6ca318cd61bde73ad671163e7a3130"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.596443 11194 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260812 01:12:01.597579 11220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.597743 11194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:12:01.597839 11194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "2e94f241a5fb4612865995df1bc22299"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "42d988199f4034ecafd5fa1e6b4a67f4"
server_key_iv: "1d6ca318cd61bde73ad671163e7a3130"
server_key_version: "encryptionkey@0"
I20260812 01:12:01.597898 11194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:01.607847 11194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:01.610399 11194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:01.610569 11194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:01.610841 11194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:01.611202 11194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:01.611263 11194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.611307 11194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:01.611336 11194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:01.618099 11194 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:39885
I20260812 01:12:01.618271 11420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:39885 every 8 connection(s)
I20260812 01:12:01.618556 11194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:12:01 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497121, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:12:01.628345 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 11194
I20260812 01:12:01.628446 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:12:01.628641 17928 external_mini_cluster.cc:1428] Setting key 68f3a233b56a1ec685ffd03441604dde
I20260812 01:12:01.631366 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.619500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:41153 (local address 127.17.130.62:46017)
0812 01:12:01.619554 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.619556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.620050 (+ 494us) server_negotiation.cc:407] Connection header received
0812 01:12:01.620654 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.620655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.620669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.620729 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:01.622580 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.623263 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.624675 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.624913 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.625009 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.627564 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:01.627566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:01.627566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:01.627575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:01.629610 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.630141 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.630143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.630143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.630217 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.630490 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.630491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.630491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.630645 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:01.630724 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.631159 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.631257 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:12:01.631603 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.619462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:41153)
0812 01:12:01.619657 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.619695 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.620531 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.620809 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.620813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.622034 (+ 1221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:01.622372 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.622384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.623616 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.623619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.624552 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.624555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.625024 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.625025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.625215 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.625849 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:01.625858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:01.627432 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:01.629708 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:01.629717 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:01.629721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:01.630037 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:01.630287 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:01.630287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:01.630288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:01.630411 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:01.630715 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:01.630722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:01.630885 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.631133 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.631328 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":94,"threads_started":1}
I20260812 01:12:01.632485 11421 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46017
I20260812 01:12:01.632592 11421 heartbeater.cc:464] Registering TS with master...
I20260812 01:12:01.632901 11421 heartbeater.cc:511] Master 127.17.130.62:46017 requested a full tablet report, sending...
I20260812 01:12:01.633533 10628 ts_manager.cc:194] Registered new tserver with Master: 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885)
I20260812 01:12:01.634087 10628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:41153
I20260812 01:12:01.641733 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:12:01.653147 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.643467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35356 (local address 127.17.130.62:46017)
0812 01:12:01.643546 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.643548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.643556 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:12:01.643580 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.643581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.643592 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.643655 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:01.644556 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.645295 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.646116 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.646319 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.646389 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.648688 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:01.648691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:01.648691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:01.648705 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:01.651071 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.651550 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.651551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.651551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.651635 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:01.651954 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:01.651955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:01.651955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:01.652160 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:01.652251 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.652566 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.652756 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:12:01.655784 10628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:12:01.657955 10628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:12:01.667825 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.662320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39885 (local address 127.0.0.1:47298)
0812 01:12:01.662779 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.662791 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.662844 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.663245 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.663246 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.663250 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:01.663485 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.663490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.664950 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.664952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.666214 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.666218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.667337 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.667338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.667411 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.667429 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.667616 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.667688 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":114,"thread_start_us":184,"threads_started":1}
I20260812 01:12:01.667960 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.662362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.17.130.1:40657)
0812 01:12:01.662990 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.663007 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.663019 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:12:01.663073 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.663082 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.663101 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.663201 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:01.663566 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.665024 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.666400 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.667076 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.667197 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.667293 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.667776 (+ 483us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.667849 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":130,"threads_started":1}
I20260812 01:12:01.668046 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.662165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:40657 (local address 127.0.0.1:39154)
0812 01:12:01.662652 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.662701 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.662785 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.663227 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.663232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.663244 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:01.663445 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.663448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.665242 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.665247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.666246 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.666250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.667202 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.667203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.667350 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.667425 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.667529 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.667621 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":7,"thread_start_us":83,"threads_started":1}
I20260812 01:12:01.668439 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.662725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47298 (local address 127.17.130.3:39885)
0812 01:12:01.663045 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.663050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.663060 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:12:01.663110 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.663112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.663131 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.663375 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:01.663600 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.664801 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.666331 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.667137 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.667224 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.667293 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.668217 (+ 924us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.668311 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":130,"threads_started":1}
I20260812 01:12:01.669459 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.662431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39631 (local address 127.0.0.1:59690)
0812 01:12:01.663017 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.663034 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.663106 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.663605 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.663606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.663609 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:01.663675 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.663676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.665098 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.665099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.667353 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.667355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.669153 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.669155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.669228 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.669234 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.669316 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.669369 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":44,"thread_start_us":116,"threads_started":1}
I20260812 01:12:01.669553 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.662558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.17.130.2:39631)
0812 01:12:01.663293 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.663301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.663313 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:12:01.663361 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.663364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.663385 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.663490 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:01.663788 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.664938 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.667478 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.668890 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.668987 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.669093 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.669359 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.669428 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":99,"threads_started":1}
I20260812 01:12:01.669521 11315 tablet_service.cc:1467] Processing CreateTablet for tablet b4d65512b4444403869f97dd0d90636d (DEFAULT_TABLE table=test-table [id=3f61ecc176454393bdfc37791d48ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:12:01.669500 10840 tablet_service.cc:1467] Processing CreateTablet for tablet b4d65512b4444403869f97dd0d90636d (DEFAULT_TABLE table=test-table [id=3f61ecc176454393bdfc37791d48ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:12:01.670018 11315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d65512b4444403869f97dd0d90636d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:01.670042 10840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d65512b4444403869f97dd0d90636d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:01.673112 11449 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Bootstrap starting.
I20260812 01:12:01.673424 11452 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Bootstrap starting.
I20260812 01:12:01.673930 11449 tablet_bootstrap.cc:654] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:01.674129 11452 tablet_bootstrap.cc:654] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:01.674327 11075 tablet_service.cc:1467] Processing CreateTablet for tablet b4d65512b4444403869f97dd0d90636d (DEFAULT_TABLE table=test-table [id=3f61ecc176454393bdfc37791d48ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:12:01.674625 11449 log.cc:824] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:01.674742 11075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4d65512b4444403869f97dd0d90636d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:01.674746 11452 log.cc:824] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:01.675571 11452 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: No bootstrap required, opened a new log
I20260812 01:12:01.675581 11449 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: No bootstrap required, opened a new log
I20260812 01:12:01.675654 11452 ts_tablet_manager.cc:1397] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:12:01.675654 11449 ts_tablet_manager.cc:1397] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:12:01.678017 11449 raft_consensus.cc:348] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.678017 11452 raft_consensus.cc:348] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.678246 11452 raft_consensus.cc:374] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:01.678246 11449 raft_consensus.cc:374] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:01.678294 11452 raft_consensus.cc:729] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4b2028807946299b80049c8b32d875, State: Initialized, Role: FOLLOWER
I20260812 01:12:01.678294 11449 raft_consensus.cc:729] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e94f241a5fb4612865995df1bc22299, State: Initialized, Role: FOLLOWER
I20260812 01:12:01.678424 11452 consensus_queue.cc:260] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.678439 11449 consensus_queue.cc:260] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.678668 11458 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Bootstrap starting.
I20260812 01:12:01.678751 11449 ts_tablet_manager.cc:1428] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260812 01:12:01.678751 11452 ts_tablet_manager.cc:1428] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:12:01.678894 11421 heartbeater.cc:503] Master 127.17.130.62:46017 was elected leader, sending a full tablet report...
I20260812 01:12:01.678912 10950 heartbeater.cc:503] Master 127.17.130.62:46017 was elected leader, sending a full tablet report...
I20260812 01:12:01.679447 11458 tablet_bootstrap.cc:654] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:01.680166 11458 log.cc:824] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:01.681021 11458 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: No bootstrap required, opened a new log
I20260812 01:12:01.681211 11458 ts_tablet_manager.cc:1397] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260812 01:12:01.682871 11458 raft_consensus.cc:348] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.683034 11458 raft_consensus.cc:374] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:01.683058 11458 raft_consensus.cc:729] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b9ea1b1490c4979a0b8eb49b97295ca, State: Initialized, Role: FOLLOWER
I20260812 01:12:01.683200 11458 consensus_queue.cc:260] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.683522 11458 ts_tablet_manager.cc:1428] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:12:01.683547 11186 heartbeater.cc:503] Master 127.17.130.62:46017 was elected leader, sending a full tablet report...
I20260812 01:12:01.720119 11461 raft_consensus.cc:482] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:12:01.720309 11461 raft_consensus.cc:504] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.720835 11461 leader_election.cc:288] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885), 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
I20260812 01:12:01.724239 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.721022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39631 (local address 127.17.130.1:45275)
0812 01:12:01.721114 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.721128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.721188 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.721421 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.721422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.721426 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:01.721510 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.721513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.722275 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.722276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.723273 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.723275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.723932 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.723933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.724029 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.724034 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.724105 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.724144 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":25}
I20260812 01:12:01.724268 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.721229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:45275 (local address 127.17.130.2:39631)
0812 01:12:01.721278 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.721281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.721290 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:12:01.721323 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.721324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.721335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.721397 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:01.721612 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.722175 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.723372 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.723842 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.723903 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.723912 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.724126 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.724176 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:12:01.724414 11448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.721219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:42641 (local address 127.17.130.3:39885)
0812 01:12:01.721269 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.721272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.721413 (+ 141us) server_negotiation.cc:407] Connection header received
0812 01:12:01.721441 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.721442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.721453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.721516 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:01.721795 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.722454 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.723443 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.723935 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.724016 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.724024 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.724271 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.724325 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:12:01.724505 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.721064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39885 (local address 127.17.130.1:42641)
0812 01:12:01.721279 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.721288 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:01.721343 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:01.721581 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:01.721582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:01.721585 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:01.721666 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.721668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.722549 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.722550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.723306 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:01.723308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.724032 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.724033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:01.724161 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.724166 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:12:01.724239 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 01:12:01.724384 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":101,"threads_started":1}
I20260812 01:12:01.724694 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d65512b4444403869f97dd0d90636d" candidate_uuid: "9f4b2028807946299b80049c8b32d875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" is_pre_election: true
I20260812 01:12:01.724864 11109 raft_consensus.cc:2454] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f4b2028807946299b80049c8b32d875 in term 0.
I20260812 01:12:01.724977 11344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d65512b4444403869f97dd0d90636d" candidate_uuid: "9f4b2028807946299b80049c8b32d875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e94f241a5fb4612865995df1bc22299" is_pre_election: true
I20260812 01:12:01.725188 11344 raft_consensus.cc:2454] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f4b2028807946299b80049c8b32d875 in term 0.
I20260812 01:12:01.725276 10763 leader_election.cc:302] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b9ea1b1490c4979a0b8eb49b97295ca, 9f4b2028807946299b80049c8b32d875; no voters:
I20260812 01:12:01.725447 11461 raft_consensus.cc:2781] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:12:01.725529 11461 raft_consensus.cc:482] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:12:01.725562 11461 raft_consensus.cc:3037] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:01.726320 11461 raft_consensus.cc:504] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.726485 11461 leader_election.cc:288] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 1 election: Requested vote from peers 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885), 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
I20260812 01:12:01.726681 11109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d65512b4444403869f97dd0d90636d" candidate_uuid: "9f4b2028807946299b80049c8b32d875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
I20260812 01:12:01.726698 11344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4d65512b4444403869f97dd0d90636d" candidate_uuid: "9f4b2028807946299b80049c8b32d875" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e94f241a5fb4612865995df1bc22299"
I20260812 01:12:01.726792 11344 raft_consensus.cc:3037] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:01.726776 11109 raft_consensus.cc:3037] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:01.727509 11344 raft_consensus.cc:2454] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f4b2028807946299b80049c8b32d875 in term 1.
I20260812 01:12:01.727743 11109 raft_consensus.cc:2454] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f4b2028807946299b80049c8b32d875 in term 1.
I20260812 01:12:01.727757 10765 leader_election.cc:302] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e94f241a5fb4612865995df1bc22299, 9f4b2028807946299b80049c8b32d875; no voters:
I20260812 01:12:01.727855 11461 raft_consensus.cc:2781] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:12:01.728009 11461 raft_consensus.cc:686] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 LEADER]: Becoming Leader. State: Replica: 9f4b2028807946299b80049c8b32d875, State: Running, Role: LEADER
I20260812 01:12:01.728119 11461 consensus_queue.cc:237] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:01.729022 10627 catalog_manager.cc:5495] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f4b2028807946299b80049c8b32d875 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "9f4b2028807946299b80049c8b32d875" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } health_report { overall_health: UNKNOWN } } }
W20260812 01:12:01.729611 10951 tablet.cc:2369] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:12:01.745654 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.742647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39166 (local address 127.17.130.1:40657)
0812 01:12:01.742724 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:01.742727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:01.742737 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:12:01.742788 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:01.742788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:01.742809 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:01.742875 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:12:01.743128 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.743744 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.744531 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:01.744735 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:01.744808 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:01.745199 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:12:01.745320 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:12:01.745409 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:01.745493 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 01:12:01.745514 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 01:12:01.747936 11344 raft_consensus.cc:1261] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Refusing update from remote peer 9f4b2028807946299b80049c8b32d875: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:12:01.747936 11109 raft_consensus.cc:1261] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Refusing update from remote peer 9f4b2028807946299b80049c8b32d875: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:12:01.748380 11472 consensus_queue.cc:1035] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:12:01.748574 11461 consensus_queue.cc:1035] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:12:01.752328 11478 mvcc.cc:205] Tried to move back new op lower bound from 7317492210675949568 to 7317492210599092224. Current Snapshot: MvccSnapshot[applied={T|T < 7317492210675949568}]
I20260812 01:12:01.753005 11476 mvcc.cc:205] Tried to move back new op lower bound from 7317492210675949568 to 7317492210599092224. Current Snapshot: MvccSnapshot[applied={T|T < 7317492210675949568}]
I20260812 01:12:01.755673 10627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:12:01.755798 10627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:12:01.756948 10627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Sending DeleteTablet for 3 replicas of tablet b4d65512b4444403869f97dd0d90636d
I20260812 01:12:01.757385 11315 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d65512b4444403869f97dd0d90636d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:01 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:47298
I20260812 01:12:01.757483 11075 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d65512b4444403869f97dd0d90636d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:01 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:59690
I20260812 01:12:01.757682 10840 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d65512b4444403869f97dd0d90636d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:01 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:39154
I20260812 01:12:01.757727 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 10734
I20260812 01:12:01.770202 11493 tablet_replica.cc:330] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: stopping tablet replica
I20260812 01:12:01.770395 11493 raft_consensus.cc:2229] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:12:01.770582 11493 raft_consensus.cc:2258] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:12:01.770808 11492 tablet_replica.cc:330] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: stopping tablet replica
I20260812 01:12:01.770946 11492 raft_consensus.cc:2229] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:12:01.771086 11492 raft_consensus.cc:2258] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:12:01.771214 11493 ts_tablet_manager.cc:1905] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:12:01.771605 11492 ts_tablet_manager.cc:1905] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:12:01.773355 11492 ts_tablet_manager.cc:1918] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:12:01.773353 11493 ts_tablet_manager.cc:1918] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:12:01.773449 11492 log.cc:1196] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/b4d65512b4444403869f97dd0d90636d
I20260812 01:12:01.773449 11493 log.cc:1196] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/b4d65512b4444403869f97dd0d90636d
I20260812 01:12:01.773773 11493 ts_tablet_manager.cc:1939] T b4d65512b4444403869f97dd0d90636d P 7b9ea1b1490c4979a0b8eb49b97295ca: Deleting consensus metadata
I20260812 01:12:01.773773 11492 ts_tablet_manager.cc:1939] T b4d65512b4444403869f97dd0d90636d P 2e94f241a5fb4612865995df1bc22299: Deleting consensus metadata
I20260812 01:12:01.774283 10601 catalog_manager.cc:4841] TS 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631): tablet b4d65512b4444403869f97dd0d90636d (table test-table [id=3f61ecc176454393bdfc37791d48ccdd]) successfully deleted
I20260812 01:12:01.774329 10603 catalog_manager.cc:4841] TS 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885): tablet b4d65512b4444403869f97dd0d90636d (table test-table [id=3f61ecc176454393bdfc37791d48ccdd]) successfully deleted
W20260812 01:12:01.778435 10601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:40657 (error 108)
I20260812 01:12:01.778558 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 10956
I20260812 01:12:01.784438 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:01.781802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:40657 (local address 127.0.0.1:39178)
0812 01:12:01.784303 (+ 2501us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:01.784374 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:40657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2464}
W20260812 01:12:01.791467 10601 catalog_manager.cc:4584] TS 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4d65512b4444403869f97dd0d90636d: Network error: Client connection negotiation failed: client connection to 127.17.130.1:40657: connect: Connection refused (error 111)
I20260812 01:12:01.796881 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 11194
I20260812 01:12:01.810497 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 10567
I20260812 01:12:21.833195 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46017
--webserver_interface=127.17.130.62
--webserver_port=44773
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:21.933579 11496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:21.933871 11496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:21.933916 11496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:21.935411 11496 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 01:12:21.935513 11496 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 01:12:21.935539 11496 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 01:12:21.935559 11496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:12:21.935590 11496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:21.935623 11496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:21.935650 11496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:12:21.935694 11496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:12:21.938195 11496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=44773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:21.938580 11496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:21.938864 11496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:21.950417 11508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:21.950466 11496 server_base.cc:1034] running on GCE node
W20260812 01:12:21.950423 11510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:21.950417 11506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:21.951244 11496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:21.951628 11496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:21.952811 11496 hybrid_clock.cc:648] HybridClock initialized: now 1786497141952770 us; error 68 us; skew 500 ppm
Aug 12 01:12:21 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497141, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:21.957580 11496 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:12:21.958307 11496 webserver.cc:460] Webserver started at http://127.17.130.62:44773/ using document root <none> and password file <none>
I20260812 01:12:21.958825 11496 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:21.958963 11496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:21.960759 11496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:12:21.962007 11520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:21.962357 11496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:21.962495 11496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "ea233c0ef0714d78b89f5d11f3cd388e"
format_stamp: "Formatted at 2026-08-12 01:12:00 on dist-test-slave-tfpf"
server_key: "d4a53edaf0463d43590c6032fd24ecaf"
server_key_iv: "4745393662fe9d9d5b2b29088382b2fd"
server_key_version: "encryptionkey@0"
I20260812 01:12:21.963025 11496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:21.995819 11496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:21.999760 11496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:21.999910 11496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:22.004894 11496 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46017
I20260812 01:12:22.004987 11600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46017 every 8 connection(s)
I20260812 01:12:22.005458 11496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:12:22.007253 11601 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:12:22.008003 11601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap starting.
I20260812 01:12:22.012773 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 11496
I20260812 01:12:22.013219 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:40657
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:46017
--webserver_port=39333
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:12:22.020088 11601 log.cc:824] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:22.022439 11601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:12:22.022874 11601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap complete.
I20260812 01:12:22.025879 11601 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:22.026185 11601 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea233c0ef0714d78b89f5d11f3cd388e, State: Initialized, Role: FOLLOWER
I20260812 01:12:22.026386 11601 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:22.026497 11601 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:12:22.026564 11601 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:12:22.026649 11601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 1 FOLLOWER]: Advancing to term 2
I20260812 01:12:22.027971 11601 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:22.028134 11601 leader_election.cc:302] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea233c0ef0714d78b89f5d11f3cd388e; no voters:
I20260812 01:12:22.028388 11601 leader_election.cc:288] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 01:12:22.028434 11608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 01:12:22.028573 11608 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 LEADER]: Becoming Leader. State: Replica: ea233c0ef0714d78b89f5d11f3cd388e, State: Running, Role: LEADER
I20260812 01:12:22.028740 11601 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:12:22.028973 11608 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:22.029533 11609 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } } }
I20260812 01:12:22.029718 11609 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: This master's current role is: LEADER
I20260812 01:12:22.029978 11610 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea233c0ef0714d78b89f5d11f3cd388e. Latest consensus state: current_term: 2 leader_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } } }
I20260812 01:12:22.030048 11610 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: This master's current role is: LEADER
I20260812 01:12:22.031519 11634 catalog_manager.cc:1259] Loaded cluster ID: 346d8fcd9eb94117b46ffbdeac7c6972
I20260812 01:12:22.031603 11634 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: loading cluster ID for follower catalog manager: success
I20260812 01:12:22.032955 11634 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: acquiring CA information for follower catalog manager: success
I20260812 01:12:22.033491 11634 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:12:22.033587 11619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:12:22.034407 11619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:12:22.034575 11619 catalog_manager.cc:1259] Loaded cluster ID: 346d8fcd9eb94117b46ffbdeac7c6972
I20260812 01:12:22.034613 11619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:12:22.034926 11619 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:12:22.035043 11619 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Loaded TSK: 0
I20260812 01:12:22.047940 11619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Generated new TSK 1
I20260812 01:12:22.048123 11619 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:12:22.123551 11605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:22.123781 11605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:22.123808 11605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:22.125121 11605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:22.125191 11605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:22.125223 11605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:12:22.126866 11605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:40657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=39333
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:22.127183 11605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:22.127457 11605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:22.133903 11645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:22.133880 11647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:22.133875 11643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:22.134402 11605 server_base.cc:1034] running on GCE node
I20260812 01:12:22.134636 11605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:22.134956 11605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:22.136168 11605 hybrid_clock.cc:648] HybridClock initialized: now 1786497142136151 us; error 30 us; skew 500 ppm
Aug 12 01:12:22 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497142, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:22.140177 11605 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:12:22.140756 11605 webserver.cc:460] Webserver started at http://127.17.130.1:39333/ using document root <none> and password file <none>
I20260812 01:12:22.141038 11605 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:22.141199 11605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:22.143069 11605 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.144204 11657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.144474 11605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.144636 11605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "9f4b2028807946299b80049c8b32d875"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "08fa3114af6f031cfa79c6ac27b2c88e"
server_key_iv: "c4371e9b30762725742467c873d8f17b"
server_key_version: "encryptionkey@0"
I20260812 01:12:22.145057 11605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:22.153409 11605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:22.156730 11605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:22.156942 11605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:22.157262 11605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:22.158150 11669 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:12:22.160360 11605 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:12:22.160457 11605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.001s
I20260812 01:12:22.160547 11605 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:12:22.161381 11605 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:12:22.161440 11605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260812 01:12:22.161520 11669 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Bootstrap starting.
I20260812 01:12:22.169305 11605 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:40657
I20260812 01:12:22.169754 11605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:12:22.173013 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 11605
I20260812 01:12:22.173405 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:39631
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:46017
--webserver_port=42247
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:12:22.173698 11669 log.cc:824] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:22.174995 11850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:40657 every 8 connection(s)
I20260812 01:12:22.176570 11669 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:12:22.176978 11669 tablet_bootstrap.cc:492] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Bootstrap complete.
I20260812 01:12:22.178761 11669 ts_tablet_manager.cc:1397] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.011s
I20260812 01:12:22.180182 11669 raft_consensus.cc:348] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:22.180936 11669 raft_consensus.cc:729] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4b2028807946299b80049c8b32d875, State: Initialized, Role: FOLLOWER
I20260812 01:12:22.181196 11669 consensus_queue.cc:260] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } } peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } }
I20260812 01:12:22.181887 11669 ts_tablet_manager.cc:1428] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
Aug 12 01:12:22 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497142, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:12:22.189822 11854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.170610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:47159)
0812 01:12:22.173370 (+ 2760us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.173443 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.174715 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.175937 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.175943 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.176378 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:22.176774 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.176783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.178540 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.178541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.179938 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.179943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.181030 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.181032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.181199 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.181865 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:22.181878 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:22.183848 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:22.187087 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:22.187107 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:22.187111 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:22.187825 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:22.188255 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:22.188256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:22.188256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:22.188431 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:22.189005 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:22.189014 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:22.189221 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.189356 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.189574 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2700,"thread_start_us":112,"threads_started":1}
I20260812 01:12:22.189844 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.170676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:47159 (local address 127.17.130.62:46017)
0812 01:12:22.175448 (+ 4772us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.175460 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.175515 (+ 55us) server_negotiation.cc:407] Connection header received
0812 01:12:22.175677 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.175682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.175727 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.175877 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:22.176945 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.178393 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.180083 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.180790 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.180884 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.184077 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:22.184083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:22.184083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:22.184099 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:22.186888 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.187982 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.187987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.187987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.188143 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.188522 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.188523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.188523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.188902 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:22.189023 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.189447 (+ 424us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.189624 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4697,"thread_start_us":81,"threads_started":1}
I20260812 01:12:22.190850 11851 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46017
I20260812 01:12:22.191071 11851 heartbeater.cc:464] Registering TS with master...
I20260812 01:12:22.191321 11851 heartbeater.cc:511] Master 127.17.130.62:46017 requested a full tablet report, sending...
I20260812 01:12:22.192210 11548 ts_manager.cc:194] Registered new tserver with Master: 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657)
I20260812 01:12:22.193570 11548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:47159
I20260812 01:12:22.195075 11851 heartbeater.cc:503] Master 127.17.130.62:46017 was elected leader, sending a full tablet report...
I20260812 01:12:22.199745 11864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.194395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.17.130.1:40657)
0812 01:12:22.194691 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.194696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.194705 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:12:22.194739 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.194741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.194898 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.195233 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:22.195282 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.196364 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.198114 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.199267 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.199368 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.199431 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.199618 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.199672 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":100,"threads_started":1}
I20260812 01:12:22.199746 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.194046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:40657 (local address 127.0.0.1:51204)
0812 01:12:22.194252 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.194294 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.194561 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.195003 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.195006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.195009 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:22.195096 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.195098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.196470 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.196471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.197935 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.197938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.199385 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.199386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.199482 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.199497 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.199583 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.199653 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":112,"threads_started":1}
I20260812 01:12:22.200424 11740 tablet_service.cc:1514] Processing DeleteTablet for tablet b4d65512b4444403869f97dd0d90636d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:01 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51204
I20260812 01:12:22.201164 11865 tablet_replica.cc:330] stopping tablet replica
I20260812 01:12:22.201289 11865 raft_consensus.cc:2229] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:12:22.201364 11865 raft_consensus.cc:2258] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:12:22.201748 11865 ts_tablet_manager.cc:1905] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:12:22.204666 11865 ts_tablet_manager.cc:1918] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:12:22.205005 11865 log.cc:1196] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal/wals/b4d65512b4444403869f97dd0d90636d
I20260812 01:12:22.205420 11865 ts_tablet_manager.cc:1939] T b4d65512b4444403869f97dd0d90636d P 9f4b2028807946299b80049c8b32d875: Deleting consensus metadata
I20260812 01:12:22.206228 11523 catalog_manager.cc:4841] TS 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657): tablet b4d65512b4444403869f97dd0d90636d (table test-table [id=3f61ecc176454393bdfc37791d48ccdd]) successfully deleted
W20260812 01:12:22.283221 11858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:22.283509 11858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:22.283556 11858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:22.284910 11858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:22.284998 11858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:22.285049 11858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:12:22.286827 11858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:39631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=42247
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:22.287168 11858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:22.287456 11858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:22.293591 11875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:22.293622 11878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:22.293622 11874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:22.293736 11858 server_base.cc:1034] running on GCE node
I20260812 01:12:22.294184 11858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:22.294510 11858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:22.295742 11858 hybrid_clock.cc:648] HybridClock initialized: now 1786497142295668 us; error 100 us; skew 500 ppm
Aug 12 01:12:22 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497142, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:22.299474 11858 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:12:22.300172 11858 webserver.cc:460] Webserver started at http://127.17.130.2:42247/ using document root <none> and password file <none>
I20260812 01:12:22.300520 11858 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:22.300622 11858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:22.302554 11858 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.303761 11889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.304215 11858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.304467 11858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "9d1a5fe6e8dd4f03d2b738b782f2cdc1"
server_key_iv: "74c01ba0060f0cc28fbd861fecdc1c4b"
server_key_version: "encryptionkey@0"
I20260812 01:12:22.305346 11858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:22.331329 11858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:22.335044 11858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:22.335294 11858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:22.335606 11858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:22.336100 11858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:22.336163 11858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.336213 11858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:22.336241 11858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.343451 11858 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:39631
I20260812 01:12:22.343525 12092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:39631 every 8 connection(s)
I20260812 01:12:22.344009 11858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
Aug 12 01:12:22 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497142, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:12:22.354235 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 11858
I20260812 01:12:22.354750 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:39885
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:46017
--webserver_port=42237
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:12:22.356781 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.345077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:55565 (local address 127.17.130.62:46017)
0812 01:12:22.345189 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.345192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.345352 (+ 160us) server_negotiation.cc:407] Connection header received
0812 01:12:22.346101 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.346103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.346115 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.346183 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:22.347280 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.347856 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.348960 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.349289 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.349377 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.352127 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:22.352135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:22.352135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:22.352145 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:22.354199 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.355230 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.355232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.355232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.355315 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.355795 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.355796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.355796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.355947 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:22.356034 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.356573 (+ 539us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.356700 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:12:22.357404 12097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.344917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:55565)
0812 01:12:22.345191 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.345245 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.345981 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.346266 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.346271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.346935 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:22.347160 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.347165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.347984 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.347985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.348855 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.348857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.349397 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.349398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.349549 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.350293 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:22.350303 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:22.351959 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:22.354393 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:22.354421 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:22.354427 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:22.355039 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:22.355431 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:22.355432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:22.355433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:22.355677 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:22.356050 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:22.356059 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:22.356323 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.356475 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.356768 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":115,"threads_started":1}
I20260812 01:12:22.358287 12093 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46017
I20260812 01:12:22.358438 12093 heartbeater.cc:464] Registering TS with master...
I20260812 01:12:22.358704 12093 heartbeater.cc:511] Master 127.17.130.62:46017 requested a full tablet report, sending...
I20260812 01:12:22.359411 11548 ts_manager.cc:194] Registered new tserver with Master: 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
I20260812 01:12:22.360020 11548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:55565
W20260812 01:12:22.455725 12102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:22.455979 12102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:22.456007 12102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:22.457404 12102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:22.457472 12102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:22.457505 12102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:12:22.459194 12102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:39885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=42237
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:22.459515 12102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:22.459795 12102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:22.465878 12116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:22.465896 12115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:22.465888 12121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:22.466343 12102 server_base.cc:1034] running on GCE node
I20260812 01:12:22.466648 12102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:22.466971 12102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:22.468153 12102 hybrid_clock.cc:648] HybridClock initialized: now 1786497142468136 us; error 31 us; skew 500 ppm
Aug 12 01:12:22 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497142, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:22.471586 12102 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:12:22.472271 12102 webserver.cc:460] Webserver started at http://127.17.130.3:42237/ using document root <none> and password file <none>
I20260812 01:12:22.472657 12102 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:22.472790 12102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:22.475208 12102 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.476364 12134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.476783 12102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.476976 12102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "2e94f241a5fb4612865995df1bc22299"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "42d988199f4034ecafd5fa1e6b4a67f4"
server_key_iv: "1d6ca318cd61bde73ad671163e7a3130"
server_key_version: "encryptionkey@0"
I20260812 01:12:22.477944 12102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:22.503290 12102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:22.506096 12102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:22.506299 12102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:22.506580 12102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:22.506948 12102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:22.507009 12102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.507055 12102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:22.507088 12102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:22.515021 12102 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:39885
I20260812 01:12:22.515072 12325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:39885 every 8 connection(s)
I20260812 01:12:22.515489 12102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:12:22 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497142, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:12:22.525426 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 12102
I20260812 01:12:22.526691 11855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.516363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:38693 (local address 127.17.130.62:46017)
0812 01:12:22.516507 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.516509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.516709 (+ 200us) server_negotiation.cc:407] Connection header received
0812 01:12:22.517341 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.517343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.517355 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.517412 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:22.518517 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.519049 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.520120 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.520324 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.520403 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.523215 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:22.523217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:22.523217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:22.523226 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:22.524731 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.525377 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.525378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.525379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.525449 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.525766 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.525767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.525768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.525938 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:22.526017 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.526354 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.526465 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260812 01:12:22.526757 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.516264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:38693)
0812 01:12:22.516547 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.516603 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.517213 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.517501 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.517513 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.518170 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:22.518385 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.518391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.519161 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.519162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.520003 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.520005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.520427 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.520428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.520609 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.521458 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:22.521468 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:22.523093 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:22.524862 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:22.524877 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:22.524880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:22.525293 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:22.525535 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:22.525536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:22.525537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:22.525685 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:22.526026 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:22.526036 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:22.526212 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.526334 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.526556 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":105,"threads_started":1}
I20260812 01:12:22.527619 12328 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46017
I20260812 01:12:22.527892 12328 heartbeater.cc:464] Registering TS with master...
I20260812 01:12:22.528199 12328 heartbeater.cc:511] Master 127.17.130.62:46017 requested a full tablet report, sending...
I20260812 01:12:22.528798 11548 ts_manager.cc:194] Registered new tserver with Master: 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885)
I20260812 01:12:22.529767 11548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:38693
I20260812 01:12:22.536521 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.525931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.17.130.62:46017)
0812 01:12:22.526097 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.526098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.526260 (+ 162us) server_negotiation.cc:407] Connection header received
0812 01:12:22.526369 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.526371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.526385 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.526471 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:22.528441 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.529535 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.530902 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.531227 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.531317 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.533059 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:22.533157 (+ 98us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:22.533158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:22.533171 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:22.534990 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.535416 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.535417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.535418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.535499 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.535763 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.535764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.535764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.535894 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:22.536075 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.536336 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.536426 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260812 01:12:22.538269 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:12:22.541723 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.538730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.17.130.62:46017)
0812 01:12:22.538774 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.538776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.538877 (+ 101us) server_negotiation.cc:407] Connection header received
0812 01:12:22.538985 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.538985 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.539019 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.539083 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:12:22.539206 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.539692 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.540776 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.541051 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.541244 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.541454 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:12:22.541510 (+ 56us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 01:12:22.541666 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43418: token expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:12:22.541870 12334 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43418: token expired
W20260812 01:12:22.542255 11430 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.17.130.62:46017)
I20260812 01:12:22.550693 12334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.542652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.17.130.62:46017)
0812 01:12:22.542714 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.542716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.542722 (+ 6us) server_negotiation.cc:407] Connection header received
0812 01:12:22.542742 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.542742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.542752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.542841 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:22.543398 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.544020 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.544879 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.545157 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.545271 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.547005 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:22.547007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:22.547008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:22.547018 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:22.549256 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.549728 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.549729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.549730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.549808 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:22.550034 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:22.550035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:22.550035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:22.550221 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:22.550345 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.550480 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.550580 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:12:22.552603 11548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:12:22.555420 11548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:12:22.559684 11740 tablet_service.cc:1467] Processing CreateTablet for tablet 6f41360176004b2cb0b8906f30b24127 (DEFAULT_TABLE table=test-table [id=af6b1f5cbbca4187917fc3a45942ad8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:12:22.559969 11740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f41360176004b2cb0b8906f30b24127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:22.561957 11669 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Bootstrap starting.
I20260812 01:12:22.562311 11669 tablet_bootstrap.cc:654] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:22.563006 11669 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: No bootstrap required, opened a new log
I20260812 01:12:22.563072 11669 ts_tablet_manager.cc:1397] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:22.563196 11669 raft_consensus.cc:348] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.563282 11669 raft_consensus.cc:374] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:22.563344 11669 raft_consensus.cc:729] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4b2028807946299b80049c8b32d875, State: Initialized, Role: FOLLOWER
I20260812 01:12:22.563410 11669 consensus_queue.cc:260] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.563520 11669 ts_tablet_manager.cc:1428] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 01:12:22.563649 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.559195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39631 (local address 127.0.0.1:54210)
0812 01:12:22.559403 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.559414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.559457 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.559964 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.559965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.559967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:22.560012 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.560014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.560988 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.560989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.562289 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.562292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.563381 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.563382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.563461 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.563467 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.563504 (+ 37us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.563555 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":24,"thread_start_us":78,"threads_started":1}
I20260812 01:12:22.563964 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.559194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39885 (local address 127.0.0.1:44000)
0812 01:12:22.559266 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.559279 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.559324 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.559866 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.559868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.559870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:22.559950 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.559952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.561539 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.561540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.562534 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.562535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.563743 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.563744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.563826 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.563829 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.563859 (+ 30us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.563907 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 01:12:22.564182 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.559409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.17.130.2:39631)
0812 01:12:22.559687 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.559691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.559704 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:12:22.559748 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.559749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.559763 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.560091 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:22.560214 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.560884 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.562404 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.563257 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.563894 (+ 637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.563983 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.564014 (+ 31us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.564067 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":80,"threads_started":1}
I20260812 01:12:22.564358 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.559378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.17.130.3:39885)
0812 01:12:22.559640 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.559646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.559656 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:12:22.559700 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.559702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.559721 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.559959 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:22.560089 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.561420 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.562657 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.563563 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.564108 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.564195 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.564222 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.564270 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":115,"threads_started":1}
I20260812 01:12:22.564921 11974 tablet_service.cc:1467] Processing CreateTablet for tablet 6f41360176004b2cb0b8906f30b24127 (DEFAULT_TABLE table=test-table [id=af6b1f5cbbca4187917fc3a45942ad8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:12:22.565001 12216 tablet_service.cc:1467] Processing CreateTablet for tablet 6f41360176004b2cb0b8906f30b24127 (DEFAULT_TABLE table=test-table [id=af6b1f5cbbca4187917fc3a45942ad8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:12:22.565372 11974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f41360176004b2cb0b8906f30b24127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:22.565472 12216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f41360176004b2cb0b8906f30b24127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:12:22.569556 12352 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Bootstrap starting.
I20260812 01:12:22.569640 12356 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Bootstrap starting.
I20260812 01:12:22.570379 12352 tablet_bootstrap.cc:654] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:22.570379 12356 tablet_bootstrap.cc:654] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Neither blocks nor log segments found. Creating new log.
I20260812 01:12:22.571076 12352 log.cc:824] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:22.571076 12356 log.cc:824] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:22.572134 12356 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: No bootstrap required, opened a new log
I20260812 01:12:22.572201 12352 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: No bootstrap required, opened a new log
I20260812 01:12:22.572252 12356 ts_tablet_manager.cc:1397] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:12:22.572276 12352 ts_tablet_manager.cc:1397] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:12:22.575227 12352 raft_consensus.cc:348] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.575227 12356 raft_consensus.cc:348] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.575467 12356 raft_consensus.cc:374] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:22.575474 12352 raft_consensus.cc:374] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:12:22.575515 12352 raft_consensus.cc:729] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b9ea1b1490c4979a0b8eb49b97295ca, State: Initialized, Role: FOLLOWER
I20260812 01:12:22.575513 12356 raft_consensus.cc:729] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e94f241a5fb4612865995df1bc22299, State: Initialized, Role: FOLLOWER
I20260812 01:12:22.575685 12352 consensus_queue.cc:260] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.575655 12356 consensus_queue.cc:260] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.576031 12352 ts_tablet_manager.cc:1428] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:12:22.576031 12356 ts_tablet_manager.cc:1428] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:12:22.576056 12093 heartbeater.cc:503] Master 127.17.130.62:46017 was elected leader, sending a full tablet report...
I20260812 01:12:22.576352 12328 heartbeater.cc:503] Master 127.17.130.62:46017 was elected leader, sending a full tablet report...
W20260812 01:12:22.594767 12096 tablet.cc:2369] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:12:22.675855 11853 tablet.cc:2369] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:12:22.766218 12329 tablet.cc:2369] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:12:22.823172 12361 raft_consensus.cc:482] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:12:22.823318 12361 raft_consensus.cc:504] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.823657 12361 leader_election.cc:288] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631), 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657)
I20260812 01:12:22.826985 12349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.824048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:60475 (local address 127.17.130.2:39631)
0812 01:12:22.824107 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.824110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.824121 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:12:22.824156 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.824157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.824168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.824256 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:22.824464 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.825234 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.826250 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.826619 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.826708 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.826718 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.826879 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.826919 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 01:12:22.826985 12333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.823920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39631 (local address 127.17.130.3:60475)
0812 01:12:22.823985 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.823999 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.824078 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.824280 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.824281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.824283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:22.824367 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.824369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.825328 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.825329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.826147 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.826150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.826706 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.826707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.826794 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.826798 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.826859 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.826903 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 01:12:22.827184 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.824180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:36847 (local address 127.17.130.1:40657)
0812 01:12:22.824343 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.824344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.824351 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:12:22.824375 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.824375 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.824385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.824480 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:12:22.824707 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.825449 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.826408 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.826792 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.826893 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.826903 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.827063 (+ 160us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.827109 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260812 01:12:22.827171 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.824061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:40657 (local address 127.17.130.3:36847)
0812 01:12:22.824262 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.824272 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:22.824313 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:22.824537 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:22.824538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:22.824541 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:12:22.824608 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.824609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.825542 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.825543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.826307 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:22.826309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.826894 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.826896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:22.826976 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.826979 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:12:22.827045 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 01:12:22.827087 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
I20260812 01:12:22.827551 11776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f41360176004b2cb0b8906f30b24127" candidate_uuid: "2e94f241a5fb4612865995df1bc22299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4b2028807946299b80049c8b32d875" is_pre_election: true
I20260812 01:12:22.827628 12008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f41360176004b2cb0b8906f30b24127" candidate_uuid: "2e94f241a5fb4612865995df1bc22299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" is_pre_election: true
I20260812 01:12:22.827754 11776 raft_consensus.cc:2454] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e94f241a5fb4612865995df1bc22299 in term 0.
I20260812 01:12:22.827806 12008 raft_consensus.cc:2454] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e94f241a5fb4612865995df1bc22299 in term 0.
I20260812 01:12:22.828032 12137 leader_election.cc:302] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e94f241a5fb4612865995df1bc22299, 9f4b2028807946299b80049c8b32d875; no voters:
I20260812 01:12:22.828168 12361 raft_consensus.cc:2781] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:12:22.828227 12361 raft_consensus.cc:482] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:12:22.828256 12361 raft_consensus.cc:3037] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:22.829170 12361 raft_consensus.cc:504] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.829311 12361 leader_election.cc:288] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [CANDIDATE]: Term 1 election: Requested vote from peers 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631), 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657)
I20260812 01:12:22.829505 12008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f41360176004b2cb0b8906f30b24127" candidate_uuid: "2e94f241a5fb4612865995df1bc22299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
I20260812 01:12:22.829504 11776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f41360176004b2cb0b8906f30b24127" candidate_uuid: "2e94f241a5fb4612865995df1bc22299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4b2028807946299b80049c8b32d875"
I20260812 01:12:22.829615 12008 raft_consensus.cc:3037] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:22.829648 11776 raft_consensus.cc:3037] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:12:22.830359 11776 raft_consensus.cc:2454] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e94f241a5fb4612865995df1bc22299 in term 1.
I20260812 01:12:22.830529 12008 raft_consensus.cc:2454] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e94f241a5fb4612865995df1bc22299 in term 1.
I20260812 01:12:22.830569 12137 leader_election.cc:302] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e94f241a5fb4612865995df1bc22299, 9f4b2028807946299b80049c8b32d875; no voters:
I20260812 01:12:22.830651 12361 raft_consensus.cc:2781] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:12:22.831064 12361 raft_consensus.cc:686] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 1 LEADER]: Becoming Leader. State: Replica: 2e94f241a5fb4612865995df1bc22299, State: Running, Role: LEADER
I20260812 01:12:22.831364 12361 consensus_queue.cc:237] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:22.832468 11548 catalog_manager.cc:5495] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e94f241a5fb4612865995df1bc22299 (127.17.130.3). New cstate: current_term: 1 leader_uuid: "2e94f241a5fb4612865995df1bc22299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } health_report { overall_health: UNKNOWN } } }
I20260812 01:12:22.853952 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.851019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.17.130.3:39885)
0812 01:12:22.851115 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:22.851117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:22.851126 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:12:22.851168 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:22.851169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:22.851188 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:22.851262 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:12:22.851487 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.851988 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.852895 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:22.853134 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:22.853218 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:22.853456 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:12:22.853568 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:12:22.853643 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:22.853833 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 01:12:22.853857 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 01:12:22.856391 12008 raft_consensus.cc:1261] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Refusing update from remote peer 2e94f241a5fb4612865995df1bc22299: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:12:22.856397 11776 raft_consensus.cc:1261] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Refusing update from remote peer 2e94f241a5fb4612865995df1bc22299: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:12:22.856793 12361 consensus_queue.cc:1035] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:12:22.857044 12373 consensus_queue.cc:1035] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:12:22.860993 12383 mvcc.cc:205] Tried to move back new op lower bound from 7317492297136193536 to 7317492297038049280. Current Snapshot: MvccSnapshot[applied={T|T < 7317492297136193536}]
I20260812 01:12:22.861858 12382 mvcc.cc:205] Tried to move back new op lower bound from 7317492297136193536 to 7317492297038049280. Current Snapshot: MvccSnapshot[applied={T|T < 7317492297136193536}]
I20260812 01:12:22.863379 11546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:12:22.863509 11546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:12:22.864503 11546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Sending DeleteTablet for 3 replicas of tablet 6f41360176004b2cb0b8906f30b24127
I20260812 01:12:22.864933 11740 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f41360176004b2cb0b8906f30b24127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:22 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:51204
I20260812 01:12:22.864966 12216 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f41360176004b2cb0b8906f30b24127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:22 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:44000
I20260812 01:12:22.865005 11974 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f41360176004b2cb0b8906f30b24127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:12:22 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:54210
I20260812 01:12:22.865245 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 11605
I20260812 01:12:22.865294 12403 tablet_replica.cc:330] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: stopping tablet replica
I20260812 01:12:22.865401 12403 raft_consensus.cc:2229] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:12:22.865511 12403 raft_consensus.cc:2258] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:12:22.883595 12403 ts_tablet_manager.cc:1905] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:12:22.885375 12402 tablet_replica.cc:330] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: stopping tablet replica
I20260812 01:12:22.885661 12402 raft_consensus.cc:2229] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:12:22.885869 12402 raft_consensus.cc:2258] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:12:22.885975 12403 ts_tablet_manager.cc:1918] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:12:22.886335 12403 log.cc:1196] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/6f41360176004b2cb0b8906f30b24127
I20260812 01:12:22.886574 12402 ts_tablet_manager.cc:1905] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:12:22.886792 12403 ts_tablet_manager.cc:1939] T 6f41360176004b2cb0b8906f30b24127 P 7b9ea1b1490c4979a0b8eb49b97295ca: Deleting consensus metadata
I20260812 01:12:22.887513 11523 catalog_manager.cc:4841] TS 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631): tablet 6f41360176004b2cb0b8906f30b24127 (table test-table [id=af6b1f5cbbca4187917fc3a45942ad8a]) successfully deleted
I20260812 01:12:22.888904 12402 ts_tablet_manager.cc:1918] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:12:22.889017 12402 log.cc:1196] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/6f41360176004b2cb0b8906f30b24127
I20260812 01:12:22.889385 12402 ts_tablet_manager.cc:1939] T 6f41360176004b2cb0b8906f30b24127 P 2e94f241a5fb4612865995df1bc22299: Deleting consensus metadata
I20260812 01:12:22.889919 11525 catalog_manager.cc:4841] TS 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885): tablet 6f41360176004b2cb0b8906f30b24127 (table test-table [id=af6b1f5cbbca4187917fc3a45942ad8a]) successfully deleted
W20260812 01:12:22.891990 11523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:40657 (error 108)
I20260812 01:12:22.892278 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 11858
I20260812 01:12:22.895514 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:22.892467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:40657 (local address 127.0.0.1:51216)
0812 01:12:22.895365 (+ 2898us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:22.895455 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:40657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2864}
W20260812 01:12:22.906330 11523 catalog_manager.cc:4584] TS 9f4b2028807946299b80049c8b32d875 (127.17.130.1:40657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f41360176004b2cb0b8906f30b24127: Network error: Client connection negotiation failed: client connection to 127.17.130.1:40657: connect: Connection refused (error 111)
I20260812 01:12:22.913290 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 12102
I20260812 01:12:22.929787 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 11496
I20260812 01:12:42.955508 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46017
--webserver_interface=127.17.130.62
--webserver_port=44773
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:43.057883 12405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:43.058225 12405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:43.058267 12405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:43.060621 12405 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 01:12:43.060719 12405 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 01:12:43.060736 12405 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 01:12:43.060750 12405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:12:43.060766 12405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:43.060781 12405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:43.060849 12405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:12:43.060873 12405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:12:43.062980 12405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=44773
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:43.063324 12405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:43.063616 12405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:43.069753 12416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.069753 12418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.069829 12414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:43.069768 12405 server_base.cc:1034] running on GCE node
I20260812 01:12:43.070269 12405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:43.070638 12405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:43.071833 12405 hybrid_clock.cc:648] HybridClock initialized: now 1786497163071821 us; error 33 us; skew 500 ppm
Aug 12 01:12:43 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:43.074793 12405 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:12:43.075255 12405 webserver.cc:460] Webserver started at http://127.17.130.62:44773/ using document root <none> and password file <none>
I20260812 01:12:43.075533 12405 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:43.075609 12405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:43.077136 12405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:43.077941 12427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.078218 12405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:43.078342 12405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "ea233c0ef0714d78b89f5d11f3cd388e"
format_stamp: "Formatted at 2026-08-12 01:12:00 on dist-test-slave-tfpf"
server_key: "d4a53edaf0463d43590c6032fd24ecaf"
server_key_iv: "4745393662fe9d9d5b2b29088382b2fd"
server_key_version: "encryptionkey@0"
I20260812 01:12:43.078738 12405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:43.086846 12405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:43.089006 12405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:43.089233 12405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:43.093914 12405 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46017
I20260812 01:12:43.093956 12520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46017 every 8 connection(s)
I20260812 01:12:43.094412 12405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:12:43.096884 12522 sys_catalog.cc:263] Verifying existing consensus state
I20260812 01:12:43.097698 12522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap starting.
I20260812 01:12:43.103653 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 12405
I20260812 01:12:43.104033 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:40657
--local_ip_for_outbound_sockets=127.17.130.1
--tserver_master_addrs=127.17.130.62:46017
--webserver_port=39333
--webserver_interface=127.17.130.1
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:12:43.108490 12522 log.cc:824] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:43.111299 12522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:12:43.111702 12522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Bootstrap complete.
I20260812 01:12:43.114284 12522 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:43.114527 12522 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea233c0ef0714d78b89f5d11f3cd388e, State: Initialized, Role: FOLLOWER
I20260812 01:12:43.114704 12522 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:43.114784 12522 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:12:43.114815 12522 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:12:43.114885 12522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 2 FOLLOWER]: Advancing to term 3
I20260812 01:12:43.116053 12522 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:43.116153 12522 leader_election.cc:302] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea233c0ef0714d78b89f5d11f3cd388e; no voters:
I20260812 01:12:43.116406 12522 leader_election.cc:288] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 01:12:43.116446 12531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 3 FOLLOWER]: Leader election won for term 3
I20260812 01:12:43.116582 12531 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [term 3 LEADER]: Becoming Leader. State: Replica: ea233c0ef0714d78b89f5d11f3cd388e, State: Running, Role: LEADER
I20260812 01:12:43.116693 12531 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } }
I20260812 01:12:43.116742 12522 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:12:43.117017 12534 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } } }
I20260812 01:12:43.117128 12534 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: This master's current role is: LEADER
I20260812 01:12:43.117394 12535 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea233c0ef0714d78b89f5d11f3cd388e. Latest consensus state: current_term: 3 leader_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea233c0ef0714d78b89f5d11f3cd388e" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46017 } } }
I20260812 01:12:43.117503 12535 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e [sys.catalog]: This master's current role is: LEADER
I20260812 01:12:43.117667 12538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:12:43.118533 12538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:12:43.118641 12538 catalog_manager.cc:1259] Loaded cluster ID: 346d8fcd9eb94117b46ffbdeac7c6972
I20260812 01:12:43.118925 12538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:12:43.120136 12538 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:12:43.120334 12538 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Loaded TSK: 0
I20260812 01:12:43.120386 12538 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ea233c0ef0714d78b89f5d11f3cd388e: Loaded TSK: 1
F20260812 01:12:43.120637 12538 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f85fd1a79ad google::LogMessage::Fail() at ??:0
@ 0x7f85fd1aa88a google::LogMessage::SendToLog() at ??:0
@ 0x7f85fd1a74de google::LogMessage::Flush() at ??:0
@ 0x7f85fd1a8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f85ff914104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f85ff911d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f86051011cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f860510148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8605101594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f86050c7c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f86050d1ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f85ff0750d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f85ff06a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f85ffece6db start_thread at ??:0
@ 0x7f85fbca561f clone at ??:0
W20260812 01:12:43.225687 12526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:43.225926 12526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:43.225967 12526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:43.227293 12526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:43.227337 12526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:43.227366 12526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:12:43.228999 12526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:40657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=39333
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:43.229341 12526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:43.229619 12526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:43.236472 12608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.236465 12606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:43.236828 12526 server_base.cc:1034] running on GCE node
W20260812 01:12:43.236472 12604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:43.237484 12526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:43.237794 12526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:43.239116 12526 hybrid_clock.cc:648] HybridClock initialized: now 1786497163239113 us; error 47 us; skew 500 ppm
Aug 12 01:12:43 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:43.242273 12526 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:12:43.242951 12526 webserver.cc:460] Webserver started at http://127.17.130.1:39333/ using document root <none> and password file <none>
I20260812 01:12:43.243259 12526 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:43.243310 12526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:43.244900 12526 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:12:43.245847 12616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.246106 12526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:12:43.246240 12526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "9f4b2028807946299b80049c8b32d875"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "08fa3114af6f031cfa79c6ac27b2c88e"
server_key_iv: "c4371e9b30762725742467c873d8f17b"
server_key_version: "encryptionkey@0"
I20260812 01:12:43.246650 12526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:43.275177 12526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:43.278555 12526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:43.278771 12526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:43.279071 12526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:43.279721 12630 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 01:12:43.282081 12526 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 01:12:43.282192 12526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260812 01:12:43.282245 12526 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 01:12:43.282967 12526 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 01:12:43.283030 12526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 01:12:43.283100 12630 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Bootstrap starting.
I20260812 01:12:43.290942 12526 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:40657
I20260812 01:12:43.291384 12526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:12:43.292129 12817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:40657 every 8 connection(s)
I20260812 01:12:43.292759 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 12526
I20260812 01:12:43.293185 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:39631
--local_ip_for_outbound_sockets=127.17.130.2
--tserver_master_addrs=127.17.130.62:46017
--webserver_port=42247
--webserver_interface=127.17.130.2
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:12:43.295048 12630 log.cc:824] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Log is configured to *not* fsync() on all Append() calls
I20260812 01:12:43.298557 12630 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 01:12:43.298899 12630 tablet_bootstrap.cc:492] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Bootstrap complete.
I20260812 01:12:43.300453 12630 ts_tablet_manager.cc:1397] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260812 01:12:43.302255 12630 raft_consensus.cc:348] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:43.302740 12630 raft_consensus.cc:729] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4b2028807946299b80049c8b32d875, State: Initialized, Role: FOLLOWER
I20260812 01:12:43.302919 12630 consensus_queue.cc:260] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:43.303337 12630 ts_tablet_manager.cc:1428] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875: Time spent starting tablet: real 0.003s user 0.003s sys 0.003s
W20260812 01:12:43.373601 12823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:12:43.292297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:42857)
0812 01:12:43.292502 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.292569 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:43.293406 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:43.373340 (+ 79934us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":118,"threads_started":1}
W20260812 01:12:43.373917 12819 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:46017 (0 consecutive failures): Network error: Failed to ping master at 127.17.130.62:46017: Client connection negotiation failed: client connection to 127.17.130.62:46017: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 01:12:43.374305 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.374164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:42039)
0812 01:12:43.374226 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.374276 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 01:12:43.374897 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.374779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:40695)
0812 01:12:43.374834 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.374862 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:12:43.375058 12819 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:12:43.412936 12827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:43.413242 12827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:43.413276 12827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:43.414573 12827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:43.414623 12827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:43.414654 12827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:12:43.416265 12827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:39631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=42247
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:43.416548 12827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:43.416862 12827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:12:43.423168 12827 server_base.cc:1034] running on GCE node
W20260812 01:12:43.423106 12840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.423183 12843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.423131 12841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:43.423704 12827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:43.423991 12827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:43.425159 12827 hybrid_clock.cc:648] HybridClock initialized: now 1786497163425111 us; error 62 us; skew 500 ppm
Aug 12 01:12:43 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:43.428009 12827 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:12:43.428575 12827 webserver.cc:460] Webserver started at http://127.17.130.2:42247/ using document root <none> and password file <none>
I20260812 01:12:43.428854 12827 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:43.428930 12827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:43.430505 12827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:43.431331 12851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.431715 12827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:12:43.431818 12827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "7b9ea1b1490c4979a0b8eb49b97295ca"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "9d1a5fe6e8dd4f03d2b738b782f2cdc1"
server_key_iv: "74c01ba0060f0cc28fbd861fecdc1c4b"
server_key_version: "encryptionkey@0"
I20260812 01:12:43.432173 12827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:43.446460 12827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:43.449496 12827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:43.449693 12827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:43.449955 12827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:43.450326 12827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:43.450387 12827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.450434 12827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:43.450462 12827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.460022 12827 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:39631
I20260812 01:12:43.460430 13082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:39631 every 8 connection(s)
I20260812 01:12:43.460697 12827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:12:43.462304 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 12827
I20260812 01:12:43.462491 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.461825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:57923)
0812 01:12:43.462119 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.462224 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":110,"threads_started":1}
I20260812 01:12:43.462669 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:39885
--local_ip_for_outbound_sockets=127.17.130.3
--tserver_master_addrs=127.17.130.62:46017
--webserver_port=42237
--webserver_interface=127.17.130.3
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:37241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:12:43.463055 13084 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:46017 (0 consecutive failures): Network error: Failed to ping master at 127.17.130.62:46017: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
I20260812 01:12:43.463590 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.463341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:53905)
0812 01:12:43.463401 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.463533 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:12:43.464160 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.464027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:38641)
0812 01:12:43.464098 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.464137 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 01:12:43.464290 13084 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 01:12:43.558521 13094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:12:43.558830 13094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:12:43.558867 13094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:12:43.560170 13094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:12:43.560235 13094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:12:43.560278 13094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:12:43.561993 13094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:37241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:39885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=42237
--tserver_master_addrs=127.17.130.62:46017
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:12:43.562294 13094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:12:43.562572 13094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:12:43.568681 13108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.568754 13112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:12:43.568912 13109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:12:43.569330 13094 server_base.cc:1034] running on GCE node
I20260812 01:12:43.569690 13094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:12:43.570086 13094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:12:43.571318 13094 hybrid_clock.cc:648] HybridClock initialized: now 1786497163571286 us; error 81 us; skew 500 ppm
Aug 12 01:12:43 dist-test-slave-tfpf krb5kdc[10551](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:12:43.574831 13094 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:12:43.575552 13094 webserver.cc:460] Webserver started at http://127.17.130.3:42237/ using document root <none> and password file <none>
I20260812 01:12:43.575842 13094 fs_manager.cc:359] Metadata directory not provided
I20260812 01:12:43.575925 13094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:12:43.577603 13094 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:12:43.578454 13126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.578727 13094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:12:43.578859 13094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "2e94f241a5fb4612865995df1bc22299"
format_stamp: "Formatted at 2026-08-12 01:12:01 on dist-test-slave-tfpf"
server_key: "42d988199f4034ecafd5fa1e6b4a67f4"
server_key_iv: "1d6ca318cd61bde73ad671163e7a3130"
server_key_version: "encryptionkey@0"
I20260812 01:12:43.579288 13094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:12:43.592361 13094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:12:43.595299 13094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:12:43.595498 13094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:12:43.595793 13094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:12:43.596143 13094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:12:43.596199 13094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.596235 13094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:12:43.596262 13094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:12:43.602682 13094 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:39885
I20260812 01:12:43.602722 13342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:39885 every 8 connection(s)
I20260812 01:12:43.603152 13094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:12:43.604321 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.603904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:32817)
0812 01:12:43.604087 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.604174 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":76,"threads_started":1}
W20260812 01:12:43.604811 13343 heartbeater.cc:644] Failed to heartbeat to 127.17.130.62:46017 (0 consecutive failures): Network error: Failed to ping master at 127.17.130.62:46017: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
I20260812 01:12:43.605376 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.605049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:59425)
0812 01:12:43.605213 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.605353 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 01:12:43.605873 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:43.605727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:34571)
0812 01:12:43.605779 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:43.605852 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:12:43.606052 13343 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 01:12:43.610760 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 13094
I20260812 01:12:44.375872 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.375587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:41967)
0812 01:12:44.375761 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:44.375800 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260812 01:12:44.464926 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.464678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:54917)
0812 01:12:44.464825 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:44.464863 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260812 01:12:44.606777 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.606509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:54505)
0812 01:12:44.606668 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:44.606715 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260812 01:12:44.786641 13354 raft_consensus.cc:482] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:12:44.786826 13354 raft_consensus.cc:504] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:44.787220 13354 leader_election.cc:288] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885), 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
Aug 12 01:12:44 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.3@KRBTEST.COM
Aug 12 01:12:44 dist-test-slave-tfpf krb5kdc[10551](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497163, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.2@KRBTEST.COM
I20260812 01:12:44.801154 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.787642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:54953 (local address 127.17.130.3:39885)
0812 01:12:44.787790 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:44.787806 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:44.787840 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:12:44.788280 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:44.788282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:44.788326 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:44.788497 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:44.789597 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.790456 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.791487 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.791919 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.792043 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:44.795145 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:44.795151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:44.795152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:44.795169 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:44.798577 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:44.799231 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:44.799234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:44.799235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:44.799356 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:44.799778 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:44.799778 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:44.799779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:44.800204 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:44.800283 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:44.800700 (+ 417us) server_negotiation.cc:299] Negotiation successful
0812 01:12:44.800860 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260812 01:12:44.801023 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.787465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:39885 (local address 127.17.130.1:54953)
0812 01:12:44.787596 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:44.787609 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:44.787668 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:44.788551 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:44.788552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:44.789275 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:44.789453 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:44.789460 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.790578 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.790579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:44.791387 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:44.791389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.792028 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.792029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:44.792160 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:44.792849 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:44.792853 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:44.794897 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:44.798706 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:44.798719 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:44.798722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:44.799117 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:44.799528 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:44.799530 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:44.799530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:44.799698 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:44.800302 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:44.800309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:44.800467 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 01:12:44.800591 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 01:12:44.800810 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1,"thread_start_us":71,"threads_started":1}
I20260812 01:12:44.801582 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.787572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:46593 (local address 127.17.130.2:39631)
0812 01:12:44.787789 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 01:12:44.787799 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:12:44.787828 (+ 29us) server_negotiation.cc:407] Connection header received
0812 01:12:44.788291 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:12:44.788293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:12:44.788329 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:12:44.788439 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:12:44.789602 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.790456 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.791487 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.791917 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.792057 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:44.795971 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:12:44.795974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:12:44.795975 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:12:44.796251 (+ 276us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:12:44.799049 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:44.799560 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:44.799564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:44.799564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:44.799671 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:12:44.800180 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:12:44.800181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:12:44.800181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:12:44.800508 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:12:44.800609 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 01:12:44.800986 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 01:12:44.801441 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":63,"threads_started":1}
I20260812 01:12:44.801590 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:44.787430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:39631 (local address 127.17.130.1:46593)
0812 01:12:44.787498 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:44.787510 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:12:44.787575 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:12:44.788528 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:12:44.788531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:12:44.789259 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:12:44.789454 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:44.789460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.790590 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.790592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:44.791388 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:12:44.791391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:12:44.792031 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:12:44.792033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:12:44.792160 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:12:44.792804 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:12:44.792813 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:12:44.795452 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:12:44.799203 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:44.799205 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:44.799206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:44.799458 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:44.799777 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:12:44.799778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:12:44.799778 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:12:44.799851 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:12:44.800627 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:12:44.800628 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:12:44.800682 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 01:12:44.800769 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 01:12:44.800869 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":59}
W20260812 01:12:44.802690 12621 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
W20260812 01:12:44.802853 12620 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
I20260812 01:12:44.802904 12620 leader_election.cc:302] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f4b2028807946299b80049c8b32d875; no voters: 2e94f241a5fb4612865995df1bc22299, 7b9ea1b1490c4979a0b8eb49b97295ca
I20260812 01:12:44.803071 13354 raft_consensus.cc:2726] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:12:45.377044 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:45.376552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:50023)
0812 01:12:45.376923 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:45.376989 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":295,"threads_started":1}
I20260812 01:12:45.465744 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:45.465479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:51639)
0812 01:12:45.465651 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:45.465692 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":75,"threads_started":1}
I20260812 01:12:45.607781 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:45.607486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:34513)
0812 01:12:45.607688 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:45.607731 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
I20260812 01:12:46.377931 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:46.377647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:57901)
0812 01:12:46.377831 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:46.377878 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":73,"threads_started":1}
I20260812 01:12:46.396813 13370 raft_consensus.cc:482] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:12:46.396965 13370 raft_consensus.cc:504] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:46.397217 13370 leader_election.cc:288] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885), 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
W20260812 01:12:46.397765 12620 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
W20260812 01:12:46.397857 12621 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
I20260812 01:12:46.397929 12620 leader_election.cc:302] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f4b2028807946299b80049c8b32d875; no voters: 2e94f241a5fb4612865995df1bc22299, 7b9ea1b1490c4979a0b8eb49b97295ca
I20260812 01:12:46.398015 13370 raft_consensus.cc:2726] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:12:46.466593 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:46.466316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:59501)
0812 01:12:46.466503 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:46.466546 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":81,"threads_started":1}
I20260812 01:12:46.608736 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:46.608427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:50087)
0812 01:12:46.608599 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:46.608674 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260812 01:12:47.379292 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:47.378918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:40723)
0812 01:12:47.379149 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:47.379220 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
I20260812 01:12:47.467350 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:47.467121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:34789)
0812 01:12:47.467265 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:47.467302 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260812 01:12:47.609697 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:47.609387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:41677)
0812 01:12:47.609600 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:47.609643 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260812 01:12:48.380388 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:48.380127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:38379)
0812 01:12:48.380287 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:48.380333 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260812 01:12:48.468302 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:48.467990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:41147)
0812 01:12:48.468206 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:48.468253 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
I20260812 01:12:48.610519 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:48.610261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:57197)
0812 01:12:48.610433 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:48.610468 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260812 01:12:49.381325 13385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:49.381038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:58179)
0812 01:12:49.381228 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:49.381275 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":114,"threads_started":1}
I20260812 01:12:49.469357 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:49.469009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:57921)
0812 01:12:49.469262 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:49.469307 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":143,"threads_started":1}
I20260812 01:12:49.597396 13388 raft_consensus.cc:482] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:12:49.597522 13388 raft_consensus.cc:504] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:49.597786 13388 leader_election.cc:288] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885), 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
W20260812 01:12:49.598524 12621 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
W20260812 01:12:49.598656 12620 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
I20260812 01:12:49.598809 12620 leader_election.cc:302] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f4b2028807946299b80049c8b32d875; no voters: 2e94f241a5fb4612865995df1bc22299, 7b9ea1b1490c4979a0b8eb49b97295ca
I20260812 01:12:49.598937 13388 raft_consensus.cc:2726] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:12:49.611714 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:49.611186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:34253)
0812 01:12:49.611588 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:49.611643 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":225,"threads_started":1}
I20260812 01:12:50.382457 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:50.382157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:51823)
0812 01:12:50.382356 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:50.382402 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260812 01:12:50.470240 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:50.469937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:58515)
0812 01:12:50.470136 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:50.470184 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260812 01:12:50.612741 13397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:50.612425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:55989)
0812 01:12:50.612625 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:50.612681 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
I20260812 01:12:51.383584 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:51.383246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:55897)
0812 01:12:51.383454 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:51.383529 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260812 01:12:51.471222 13400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:51.470904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:53613)
0812 01:12:51.471121 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:51.471169 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":95,"threads_started":1}
I20260812 01:12:51.613870 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:51.613578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:43401)
0812 01:12:51.613766 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:51.613816 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":93,"threads_started":1}
I20260812 01:12:51.851739 13402 raft_consensus.cc:482] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:12:51.851861 13402 raft_consensus.cc:504] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e94f241a5fb4612865995df1bc22299" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 39885 } } peers { permanent_uuid: "7b9ea1b1490c4979a0b8eb49b97295ca" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 39631 } } peers { permanent_uuid: "9f4b2028807946299b80049c8b32d875" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 40657 } }
I20260812 01:12:51.852090 13402 leader_election.cc:288] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885), 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631)
W20260812 01:12:51.852552 12621 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2e94f241a5fb4612865995df1bc22299 (127.17.130.3:39885): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
W20260812 01:12:51.852648 12620 leader_election.cc:341] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7b9ea1b1490c4979a0b8eb49b97295ca (127.17.130.2:39631): Not found: Tablet not found: 6f41360176004b2cb0b8906f30b24127
I20260812 01:12:51.852727 12620 leader_election.cc:302] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f4b2028807946299b80049c8b32d875; no voters: 2e94f241a5fb4612865995df1bc22299, 7b9ea1b1490c4979a0b8eb49b97295ca
I20260812 01:12:51.852823 13402 raft_consensus.cc:2726] T 6f41360176004b2cb0b8906f30b24127 P 9f4b2028807946299b80049c8b32d875 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 01:12:52.384635 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:52.384336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:51191)
0812 01:12:52.384527 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:52.384579 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":79,"threads_started":1}
I20260812 01:12:52.472420 13406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:52.471906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:48273)
0812 01:12:52.472140 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:52.472283 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260812 01:12:52.614907 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:52.614632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:35195)
0812 01:12:52.614806 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:52.614852 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260812 01:12:53.385599 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:53.385329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:57991)
0812 01:12:53.385500 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:53.385548 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":71,"threads_started":1}
I20260812 01:12:53.473465 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:53.473220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:41803)
0812 01:12:53.473372 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:53.473410 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":65,"threads_started":1}
I20260812 01:12:53.615943 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:53.615636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:53401)
0812 01:12:53.615834 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:53.615880 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":102,"threads_started":1}
I20260812 01:12:54.388201 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:54.386212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:41971)
0812 01:12:54.388093 (+ 1881us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:54.388138 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1856,"thread_start_us":1783,"threads_started":1}
I20260812 01:12:54.474429 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:54.474158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:57425)
0812 01:12:54.474331 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:54.474378 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260812 01:12:54.617051 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:54.616753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:36931)
0812 01:12:54.616929 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:54.616984 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260812 01:12:55.388908 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:55.388682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:47107)
0812 01:12:55.388821 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:55.388860 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260812 01:12:55.475409 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:55.475066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:46875)
0812 01:12:55.475304 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:55.475356 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":137,"threads_started":1}
I20260812 01:12:55.618238 13431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:55.617981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:57013)
0812 01:12:55.618148 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:55.618186 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260812 01:12:56.389829 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:56.389564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:60971)
0812 01:12:56.389742 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:56.389779 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260812 01:12:56.476336 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:56.476060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:39329)
0812 01:12:56.476232 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:56.476285 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":67,"threads_started":1}
I20260812 01:12:56.619081 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:56.618790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:60831)
0812 01:12:56.618983 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:56.619026 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
I20260812 01:12:57.390533 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:57.390295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:34671)
0812 01:12:57.390444 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:57.390483 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260812 01:12:57.477231 13441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:57.476900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:60643)
0812 01:12:57.477050 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:57.477177 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260812 01:12:57.619939 13442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:57.619672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:33001)
0812 01:12:57.619843 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:57.619888 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260812 01:12:58.391458 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:58.391168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.1:51099)
0812 01:12:58.391343 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:58.391396 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260812 01:12:58.478308 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:58.477984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.2:53495)
0812 01:12:58.478187 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:58.478253 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
I20260812 01:12:58.620982 13446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:12:58.620659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46017 (local address 127.17.130.3:36567)
0812 01:12:58.620882 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:12:58.620931 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.62:46017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
I20260812 01:12:58.621907 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 12526
I20260812 01:12:58.642495 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 12827
I20260812 01:12:58.660348 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 13094
2026-08-12T01:12:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60095 ms)
[----------] 30 tests from SecurityITest (260537 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:12:59 dist-test-slave-tfpf krb5kdc[13451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:12:59 dist-test-slave-tfpf krb5kdc[13451](info): set up 2 sockets
Aug 12 01:12:59 dist-test-slave-tfpf krb5kdc[13451](info): commencing operation
krb5kdc: starting...
W20260812 01:13:00.799656 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.391s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:00 dist-test-slave-tfpf krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:00.969151 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:37971
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:37971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:01.064284 13470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:01.064534 13470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:01.064563 13470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:01.065786 13470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:01.065833 13470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:01.065848 13470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:01.065860 13470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:01.065874 13470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:01.067472 13470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:37971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:37971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:01.067718 13470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:01.067917 13470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:01.073482 13484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:01.073549 13470 server_base.cc:1034] running on GCE node
W20260812 01:13:01.073482 13480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:01.073547 13481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:01.074057 13470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:01.074410 13470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:01.075593 13470 hybrid_clock.cc:648] HybridClock initialized: now 1786497181075567 us; error 52 us; skew 500 ppm
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:01.078557 13470 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:01.079099 13470 webserver.cc:460] Webserver started at http://127.17.130.62:38049/ using document root <none> and password file <none>
I20260812 01:13:01.079351 13470 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:01.079419 13470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:01.079659 13470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:01.080896 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "a7b5118f6eb5456b9323992e5413c398"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "379e2130d8a840cd1d1c4b4bd4b887f1"
server_key_iv: "ccfcdba85bad226c7697c91617fbfd48"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.081530 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "a7b5118f6eb5456b9323992e5413c398"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "379e2130d8a840cd1d1c4b4bd4b887f1"
server_key_iv: "ccfcdba85bad226c7697c91617fbfd48"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.083308 13470 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:01.084375 13501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.084631 13470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:01.084760 13470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "a7b5118f6eb5456b9323992e5413c398"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "379e2130d8a840cd1d1c4b4bd4b887f1"
server_key_iv: "ccfcdba85bad226c7697c91617fbfd48"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.084849 13470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:01.095170 13470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:01.097744 13470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:01.097916 13470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:01.102875 13470 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:37971
I20260812 01:13:01.102967 13602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:37971 every 8 connection(s)
I20260812 01:13:01.103396 13470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:01.104427 13603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:01.106985 13603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: Bootstrap starting.
I20260812 01:13:01.107005 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 13470
I20260812 01:13:01.107139 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:01.107342 17928 external_mini_cluster.cc:1428] Setting key 1db40b1af2826ae737366161fe92addb
I20260812 01:13:01.107712 13603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:01.108124 13603 log.cc:824] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:01.108940 13603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: No bootstrap required, opened a new log
I20260812 01:13:01.111439 13603 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5118f6eb5456b9323992e5413c398" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37971 } }
I20260812 01:13:01.111582 13603 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:01.111606 13603 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7b5118f6eb5456b9323992e5413c398, State: Initialized, Role: FOLLOWER
I20260812 01:13:01.111738 13603 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5118f6eb5456b9323992e5413c398" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37971 } }
I20260812 01:13:01.111810 13603 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:01.111850 13603 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:01.111902 13603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:01.112591 13603 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5118f6eb5456b9323992e5413c398" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37971 } }
I20260812 01:13:01.112703 13603 leader_election.cc:302] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7b5118f6eb5456b9323992e5413c398; no voters:
I20260812 01:13:01.112864 13603 leader_election.cc:288] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:01.112990 13613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:01.113195 13603 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:01.113207 13613 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [term 1 LEADER]: Becoming Leader. State: Replica: a7b5118f6eb5456b9323992e5413c398, State: Running, Role: LEADER
I20260812 01:13:01.113343 13613 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5118f6eb5456b9323992e5413c398" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37971 } }
I20260812 01:13:01.114653 13615 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7b5118f6eb5456b9323992e5413c398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5118f6eb5456b9323992e5413c398" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37971 } } }
I20260812 01:13:01.115000 13615 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:01.115363 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7b5118f6eb5456b9323992e5413c398. Latest consensus state: current_term: 1 leader_uuid: "a7b5118f6eb5456b9323992e5413c398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7b5118f6eb5456b9323992e5413c398" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 37971 } } }
W20260812 01:13:01.115403 13633 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 01:13:01.115460 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398 [sys.catalog]: This master's current role is: LEADER
W20260812 01:13:01.115473 13633 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:01.115900 13638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:01.116149 13638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:01.118054 13638 catalog_manager.cc:1347] Generated new cluster ID: 11f0f7c045014cc7b5642d4b38854cea
I20260812 01:13:01.118120 13638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:01.120594 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.107966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.17.130.62:37971)
0812 01:13:01.108195 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.108205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.108239 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:13:01.108828 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.108837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.108901 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.109153 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:01.109764 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.111243 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.112313 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.112963 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.113438 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.116336 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:01.116341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:01.116341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:01.116356 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:01.118676 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.119154 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.119157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.119157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.119296 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.119524 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.119525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.119525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.119867 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:01.120049 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.120181 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.120329 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":124,"threads_started":1}
I20260812 01:13:01.123517 13638 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:01.124209 13638 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:01.130520 13638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: Generated new TSK 0
I20260812 01:13:01.130754 13638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:13:01.168072 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37971
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:01.270459 13642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:01.270692 13642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:01.270720 13642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:01.271919 13642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:01.271972 13642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:01.272001 13642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:13:01.273571 13642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:01.273845 13642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:01.274125 13642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:13:01.281157 13642 server_base.cc:1034] running on GCE node
W20260812 01:13:01.281116 13652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:01.281176 13654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:01.281136 13656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:01.281530 13642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:01.281804 13642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:01.283058 13642 hybrid_clock.cc:648] HybridClock initialized: now 1786497181283017 us; error 67 us; skew 500 ppm
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:01.286217 13642 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:13:01.286787 13642 webserver.cc:460] Webserver started at http://127.17.130.1:37039/ using document root <none> and password file <none>
I20260812 01:13:01.287047 13642 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:01.287123 13642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:01.287283 13642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:01.288723 13642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "e4b5066de6b1476088ea14c638be6864"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "0c351c37989aea3bca05768ec477700b"
server_key_iv: "20d24e5bac72609b2eee5de5e6ef9950"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.289187 13642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "e4b5066de6b1476088ea14c638be6864"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "0c351c37989aea3bca05768ec477700b"
server_key_iv: "20d24e5bac72609b2eee5de5e6ef9950"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.291013 13642 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:01.291980 13673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.292271 13642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:01.292407 13642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "e4b5066de6b1476088ea14c638be6864"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "0c351c37989aea3bca05768ec477700b"
server_key_iv: "20d24e5bac72609b2eee5de5e6ef9950"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.292493 13642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:01.317427 13642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:01.320387 13642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:01.320613 13642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:01.320899 13642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:01.321384 13642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:01.321444 13642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.321485 13642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:01.321518 13642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.329834 13642 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:34511
I20260812 01:13:01.329888 13879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:34511 every 8 connection(s)
I20260812 01:13:01.330289 13642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:13:01.336050 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 13642
I20260812 01:13:01.336193 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:13:01.336577 17928 external_mini_cluster.cc:1428] Setting key 261f361db2b0c011e02f5ca4ee5d5a21
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:01.343016 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.331132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:51437 (local address 127.17.130.62:37971)
0812 01:13:01.331194 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.331197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.331433 (+ 236us) server_negotiation.cc:407] Connection header received
0812 01:13:01.331997 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.331998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.332011 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.332101 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:01.333236 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.333762 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.334958 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.335275 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.335355 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.338789 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:01.338791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:01.338791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:01.338803 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:01.341247 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.341842 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.341843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.341844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.341907 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.342235 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.342236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.342236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.342363 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:01.342416 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.342827 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.342930 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:13:01.343284 13885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.331088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:37971 (local address 127.17.130.1:51437)
0812 01:13:01.331277 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:01.331325 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:01.331892 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:01.332165 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:01.332170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:01.332867 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:01.333125 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.333130 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.333886 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.333887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.334837 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.334840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.335406 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.335408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.335547 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.336291 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:01.336304 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:01.338556 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:01.341353 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:01.341363 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:01.341367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:01.341756 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:01.341985 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:01.341986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:01.341986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:01.342155 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:01.342429 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:01.342437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:01.342638 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 01:13:01.342764 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 01:13:01.342977 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260812 01:13:01.344076 13882 heartbeater.cc:347] Connected to a master server at 127.17.130.62:37971
I20260812 01:13:01.344225 13882 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:01.344436 13882 heartbeater.cc:511] Master 127.17.130.62:37971 requested a full tablet report, sending...
I20260812 01:13:01.345320 13536 ts_manager.cc:194] Registered new tserver with Master: e4b5066de6b1476088ea14c638be6864 (127.17.130.1:34511)
I20260812 01:13:01.346145 13536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:51437
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:13:01.370002 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37971
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:01.465001 13889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:01.465327 13889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:01.465371 13889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:01.466562 13889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:01.466634 13889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:01.466681 13889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:13:01.468202 13889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:01.468482 13889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:01.468751 13889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:01.474205 13900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:01.474211 13896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:01.474221 13908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:01.474501 13889 server_base.cc:1034] running on GCE node
I20260812 01:13:01.474714 13889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:01.475006 13889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:01.476197 13889 hybrid_clock.cc:648] HybridClock initialized: now 1786497181476184 us; error 34 us; skew 500 ppm
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:01.479182 13889 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:13:01.479729 13889 webserver.cc:460] Webserver started at http://127.17.130.2:46443/ using document root <none> and password file <none>
I20260812 01:13:01.479985 13889 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:01.480062 13889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:01.480224 13889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:01.481511 13889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "e9f11676964d4beeaa1c12e18c18bad8"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "f2c8f65f5cb6d671e059aab6ee39cdee"
server_key_iv: "fc9a7e24c35254c2dc8fa8bfb6317043"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.481918 13889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "e9f11676964d4beeaa1c12e18c18bad8"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "f2c8f65f5cb6d671e059aab6ee39cdee"
server_key_iv: "fc9a7e24c35254c2dc8fa8bfb6317043"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.483780 13889 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:01.485203 13920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.485389 13889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:01.485524 13889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "e9f11676964d4beeaa1c12e18c18bad8"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "f2c8f65f5cb6d671e059aab6ee39cdee"
server_key_iv: "fc9a7e24c35254c2dc8fa8bfb6317043"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.485630 13889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:01.504983 13889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:01.507551 13889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:01.507680 13889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:01.507920 13889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:01.508266 13889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:01.508302 13889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.508344 13889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:01.508374 13889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.514987 13889 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:38071
I20260812 01:13:01.515025 14125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:38071 every 8 connection(s)
I20260812 01:13:01.515410 13889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:13:01.517817 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 13889
I20260812 01:13:01.517970 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:13:01.518187 17928 external_mini_cluster.cc:1428] Setting key d8e2dc75769cfc5bca73809cc413e7c4
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:01.528602 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.516316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:39497 (local address 127.17.130.62:37971)
0812 01:13:01.516389 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.516391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.516567 (+ 176us) server_negotiation.cc:407] Connection header received
0812 01:13:01.517689 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.517691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.517708 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.517765 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:01.519265 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.519786 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.520893 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.521228 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.521296 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.524835 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:01.524836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:01.524837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:01.524845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:01.526886 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.527423 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.527424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.527425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.527491 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.527759 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.527760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.527760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.527892 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:01.527969 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.528387 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.528488 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:13:01.528821 14129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.516197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:37971 (local address 127.17.130.2:39497)
0812 01:13:01.516364 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:01.516408 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:01.517449 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:01.517847 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:01.517850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:01.518876 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:01.519158 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.519165 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.519889 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.519890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.520710 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.520712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.521334 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.521335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.521493 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.522211 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:01.522223 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:01.524691 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:01.527014 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:01.527024 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:01.527027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:01.527321 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:01.527560 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:01.527560 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:01.527561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:01.527685 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:01.527966 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:01.527974 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:01.528200 (+ 226us) client_negotiation.cc:769] Sending connection context
0812 01:13:01.528358 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 01:13:01.528555 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260812 01:13:01.529438 14127 heartbeater.cc:347] Connected to a master server at 127.17.130.62:37971
I20260812 01:13:01.529534 14127 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:01.529744 14127 heartbeater.cc:511] Master 127.17.130.62:37971 requested a full tablet report, sending...
I20260812 01:13:01.530225 13536 ts_manager.cc:194] Registered new tserver with Master: e9f11676964d4beeaa1c12e18c18bad8 (127.17.130.2:38071)
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:13:01.530836 13536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:39497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:13:01.553319 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37971
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:40829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:01.648720 14136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:01.648962 14136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:01.648990 14136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:01.650338 14136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:01.650393 14136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:01.650424 14136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:13:01.651968 14136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:40829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:37971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:01.652244 14136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:01.652467 14136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:01.658459 14142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:01.658453 14148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:01.658581 14136 server_base.cc:1034] running on GCE node
W20260812 01:13:01.658459 14145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:01.658981 14136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:01.659286 14136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:01.660511 14136 hybrid_clock.cc:648] HybridClock initialized: now 1786497181660492 us; error 44 us; skew 500 ppm
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:01.664355 14136 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:13:01.664911 14136 webserver.cc:460] Webserver started at http://127.17.130.3:40973/ using document root <none> and password file <none>
I20260812 01:13:01.665239 14136 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:01.665315 14136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:01.665485 14136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:01.666741 14136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "ced9a6a46a3544e9b4c186f02b721a19"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "7538784ef244da72c48e835d0de91eda"
server_key_iv: "ad5179c9d4e8f4efa8247641009fc311"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.667167 14136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "ced9a6a46a3544e9b4c186f02b721a19"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "7538784ef244da72c48e835d0de91eda"
server_key_iv: "ad5179c9d4e8f4efa8247641009fc311"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.668787 14136 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:13:01.669953 14165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.670198 14136 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:01.670290 14136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "ced9a6a46a3544e9b4c186f02b721a19"
format_stamp: "Formatted at 2026-08-12 01:13:01 on dist-test-slave-tfpf"
server_key: "7538784ef244da72c48e835d0de91eda"
server_key_iv: "ad5179c9d4e8f4efa8247641009fc311"
server_key_version: "encryptionkey@0"
I20260812 01:13:01.670392 14136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:01.693054 14136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:01.695653 14136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:01.695798 14136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:01.696105 14136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:01.696610 14136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:01.696653 14136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.696688 14136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:01.696702 14136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:01.703336 14136 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:36751
I20260812 01:13:01.703449 14344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:36751 every 8 connection(s)
I20260812 01:13:01.703764 14136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
Aug 12 01:13:01 dist-test-slave-tfpf krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497181, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:01.710726 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 14136
I20260812 01:13:01.710847 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:13:01.711063 17928 external_mini_cluster.cc:1428] Setting key 5f125264d86ef058eea4a97727c334f0
I20260812 01:13:01.714907 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.704717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:51281 (local address 127.17.130.62:37971)
0812 01:13:01.704769 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.704771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.704880 (+ 109us) server_negotiation.cc:407] Connection header received
0812 01:13:01.705461 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.705462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.705476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.705542 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:01.706596 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.707110 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.708153 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.708352 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.708428 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.711425 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:01.711427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:01.711427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:01.711437 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:01.713332 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.713816 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.713817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.713818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.713885 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.714173 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.714174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.714175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.714329 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:01.714405 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.714692 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.714782 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:01.715236 14347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.704553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:37971 (local address 127.17.130.3:51281)
0812 01:13:01.704743 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:01.704788 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:01.705348 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:01.705619 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:01.705624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:01.706272 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:01.706481 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.706487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.707212 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.707214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.708046 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.708049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.708438 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.708439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.708576 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.709250 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:01.709260 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:01.711258 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:01.713453 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:01.713462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:01.713465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:01.713725 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:01.713954 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:01.713955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:01.713955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:01.714099 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:01.714395 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:01.714402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:01.714553 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 01:13:01.714672 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 01:13:01.714853 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260812 01:13:01.715747 14345 heartbeater.cc:347] Connected to a master server at 127.17.130.62:37971
I20260812 01:13:01.715919 14345 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:01.716172 14345 heartbeater.cc:511] Master 127.17.130.62:37971 requested a full tablet report, sending...
I20260812 01:13:01.716836 13536 ts_manager.cc:194] Registered new tserver with Master: ced9a6a46a3544e9b4c186f02b721a19 (127.17.130.3:36751)
I20260812 01:13:01.717379 13536 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:51281
I20260812 01:13:01.723405 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:13:01.733122 13608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.724877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.17.130.62:37971)
0812 01:13:01.724949 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.724951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.725149 (+ 198us) server_negotiation.cc:407] Connection header received
0812 01:13:01.725240 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.725241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.725252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.725312 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:01.726049 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.726637 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.727572 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.727900 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.728005 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.729814 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:01.729816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:01.729817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:01.729828 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:01.731332 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.731737 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.731737 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.731738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.731792 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:01.732086 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:01.732089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:01.732089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:01.732385 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:01.732523 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.732802 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.732919 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 01:13:01.735474 13536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45402:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:13:01.738059 13536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:13:01.745124 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.741301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34511 (local address 127.0.0.1:50006)
0812 01:13:01.741588 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:01.741597 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:01.741641 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:01.741899 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:01.741903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:01.741906 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:01.742025 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.742027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.743028 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.743029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.743908 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.743910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.744769 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.744770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.744858 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.744874 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:13:01.744949 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:13:01.745016 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":126,"threads_started":1}
I20260812 01:13:01.745132 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.741432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.17.130.1:34511)
0812 01:13:01.741654 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.741661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.741703 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:13:01.741750 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.741752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.741771 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.742135 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:01.742218 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.742908 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.744022 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.744681 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.744769 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.744834 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.744973 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.745016 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":101,"threads_started":1}
I20260812 01:13:01.746352 13754 tablet_service.cc:1467] Processing CreateTablet for tablet a69b384287b84b27a844f6da83a069a8 (DEFAULT_TABLE table=test-table [id=c7010cad6f4a48eb8d34e69bb60c33a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:01.746824 13754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69b384287b84b27a844f6da83a069a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:01.747028 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.741527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36751 (local address 127.0.0.1:58272)
0812 01:13:01.741759 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:01.741768 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:01.741812 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:01.742152 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:01.742153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:01.742155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:01.742239 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.742241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.744288 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.744288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.745542 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.745546 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.746757 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.746758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.746825 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.746830 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:01.746893 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 01:13:01.746950 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":28,"thread_start_us":72,"threads_started":1}
I20260812 01:13:01.747090 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.741748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.17.130.3:36751)
0812 01:13:01.741956 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.741960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.741969 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:01.742008 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.742010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.742026 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.742310 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:01.742392 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.744127 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.745666 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.746444 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.746544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.746654 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.746944 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.747004 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
I20260812 01:13:01.747440 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.741238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:38071 (local address 127.0.0.1:60056)
0812 01:13:01.741514 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:01.741561 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:01.741640 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:01.742154 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:01.742155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:01.742156 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:01.742261 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.742263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.744277 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.744278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.745544 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:01.745546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.747196 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.747197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:01.747277 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.747281 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:01.747333 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 01:13:01.747378 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":106,"thread_start_us":111,"threads_started":1}
I20260812 01:13:01.747932 14249 tablet_service.cc:1467] Processing CreateTablet for tablet a69b384287b84b27a844f6da83a069a8 (DEFAULT_TABLE table=test-table [id=c7010cad6f4a48eb8d34e69bb60c33a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:01.748271 14249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69b384287b84b27a844f6da83a069a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:01.750669 14376 tablet_bootstrap.cc:492] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Bootstrap starting.
I20260812 01:13:01.751310 14376 tablet_bootstrap.cc:654] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:01.751511 14379 tablet_bootstrap.cc:492] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Bootstrap starting.
I20260812 01:13:01.751742 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:01.741381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60056 (local address 127.17.130.2:38071)
0812 01:13:01.741797 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:01.741802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:01.741813 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:13:01.742013 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:01.742015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:01.742032 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:01.742339 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:01.742406 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.743806 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.745666 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:01.746444 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:01.746544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:01.746654 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:01.751590 (+ 4936us) server_negotiation.cc:299] Negotiation successful
0812 01:13:01.751659 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":127,"threads_started":1}
I20260812 01:13:01.752090 14376 log.cc:824] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:01.752239 14379 tablet_bootstrap.cc:654] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:01.752451 14013 tablet_service.cc:1467] Processing CreateTablet for tablet a69b384287b84b27a844f6da83a069a8 (DEFAULT_TABLE table=test-table [id=c7010cad6f4a48eb8d34e69bb60c33a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:01.752750 14379 log.cc:824] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:01.752782 14013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a69b384287b84b27a844f6da83a069a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:01.753531 14379 tablet_bootstrap.cc:492] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: No bootstrap required, opened a new log
I20260812 01:13:01.753624 14379 ts_tablet_manager.cc:1397] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:01.754980 14379 raft_consensus.cc:348] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:01.755373 14379 raft_consensus.cc:374] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:01.755438 14379 raft_consensus.cc:729] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ced9a6a46a3544e9b4c186f02b721a19, State: Initialized, Role: FOLLOWER
I20260812 01:13:01.755617 14379 consensus_queue.cc:260] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:01.755921 14389 tablet_bootstrap.cc:492] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Bootstrap starting.
I20260812 01:13:01.755954 14379 ts_tablet_manager.cc:1428] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:13:01.756085 14345 heartbeater.cc:503] Master 127.17.130.62:37971 was elected leader, sending a full tablet report...
I20260812 01:13:01.756436 14376 tablet_bootstrap.cc:492] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: No bootstrap required, opened a new log
I20260812 01:13:01.756531 14376 ts_tablet_manager.cc:1397] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 01:13:01.756702 14389 tablet_bootstrap.cc:654] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:01.757342 14389 log.cc:824] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:01.758082 14389 tablet_bootstrap.cc:492] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: No bootstrap required, opened a new log
I20260812 01:13:01.758138 14389 ts_tablet_manager.cc:1397] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:01.758455 14376 raft_consensus.cc:348] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:01.758622 14376 raft_consensus.cc:374] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:01.758667 14376 raft_consensus.cc:729] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4b5066de6b1476088ea14c638be6864, State: Initialized, Role: FOLLOWER
I20260812 01:13:01.758800 14376 consensus_queue.cc:260] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:01.759106 14376 ts_tablet_manager.cc:1428] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:01.759250 13882 heartbeater.cc:503] Master 127.17.130.62:37971 was elected leader, sending a full tablet report...
I20260812 01:13:01.760381 14389 raft_consensus.cc:348] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:01.760677 14389 raft_consensus.cc:374] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:01.760736 14389 raft_consensus.cc:729] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9f11676964d4beeaa1c12e18c18bad8, State: Initialized, Role: FOLLOWER
I20260812 01:13:01.760934 14389 consensus_queue.cc:260] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:01.761466 14389 ts_tablet_manager.cc:1428] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 01:13:01.761509 14127 heartbeater.cc:503] Master 127.17.130.62:37971 was elected leader, sending a full tablet report...
W20260812 01:13:01.766073 14128 tablet.cc:2369] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:13:01.831475 13883 tablet.cc:2369] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:13:01.954457 14346 tablet.cc:2369] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:02.059321 14399 raft_consensus.cc:482] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:13:02.059615 14399 raft_consensus.cc:504] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:02.060333 14399 leader_election.cc:288] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9f11676964d4beeaa1c12e18c18bad8 (127.17.130.2:38071), ced9a6a46a3544e9b4c186f02b721a19 (127.17.130.3:36751)
I20260812 01:13:02.064437 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.060663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:38071 (local address 127.17.130.1:48939)
0812 01:13:02.060988 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.061007 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.061154 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.061371 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.061372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.061374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:02.061469 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.061471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.062638 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.062639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.063624 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.063627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.064185 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.064186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.064265 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.064270 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.064325 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.064361 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":242,"threads_started":1}
I20260812 01:13:02.064572 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.060911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:48939 (local address 127.17.130.2:38071)
0812 01:13:02.061011 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.061013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.061128 (+ 115us) server_negotiation.cc:407] Connection header received
0812 01:13:02.061268 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.061270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.061285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.061353 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:02.061630 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.062524 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.063730 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.064095 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.064158 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.064166 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.064449 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.064499 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 01:13:02.064646 14402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.061238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36751 (local address 127.17.130.1:44607)
0812 01:13:02.061417 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.061436 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.061580 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.061825 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.061827 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.061830 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:02.062011 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.062015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.062775 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.062776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.063645 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.063647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.064261 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.064262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.064442 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.064447 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.064515 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.064554 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":55,"thread_start_us":111,"threads_started":1}
I20260812 01:13:02.064666 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.061287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:44607 (local address 127.17.130.3:36751)
0812 01:13:02.061349 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.061351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.061499 (+ 148us) server_negotiation.cc:407] Connection header received
0812 01:13:02.061731 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.061732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.061741 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.061794 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:02.062173 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.062673 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.063757 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.064170 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.064247 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.064257 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.064528 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.064584 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 01:13:02.064936 14044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69b384287b84b27a844f6da83a069a8" candidate_uuid: "e4b5066de6b1476088ea14c638be6864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9f11676964d4beeaa1c12e18c18bad8" is_pre_election: true
I20260812 01:13:02.065032 14274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69b384287b84b27a844f6da83a069a8" candidate_uuid: "e4b5066de6b1476088ea14c638be6864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced9a6a46a3544e9b4c186f02b721a19" is_pre_election: true
I20260812 01:13:02.065151 14044 raft_consensus.cc:2454] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4b5066de6b1476088ea14c638be6864 in term 0.
I20260812 01:13:02.065230 14274 raft_consensus.cc:2454] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4b5066de6b1476088ea14c638be6864 in term 0.
I20260812 01:13:02.065465 13678 leader_election.cc:302] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4b5066de6b1476088ea14c638be6864, e9f11676964d4beeaa1c12e18c18bad8; no voters:
I20260812 01:13:02.065608 14399 raft_consensus.cc:2781] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:13:02.065671 14399 raft_consensus.cc:482] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:13:02.065704 14399 raft_consensus.cc:3037] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:02.066493 14399 raft_consensus.cc:504] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:02.066655 14399 leader_election.cc:288] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [CANDIDATE]: Term 1 election: Requested vote from peers e9f11676964d4beeaa1c12e18c18bad8 (127.17.130.2:38071), ced9a6a46a3544e9b4c186f02b721a19 (127.17.130.3:36751)
I20260812 01:13:02.066841 14044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69b384287b84b27a844f6da83a069a8" candidate_uuid: "e4b5066de6b1476088ea14c638be6864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9f11676964d4beeaa1c12e18c18bad8"
I20260812 01:13:02.066968 14044 raft_consensus.cc:3037] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:02.066951 14274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a69b384287b84b27a844f6da83a069a8" candidate_uuid: "e4b5066de6b1476088ea14c638be6864" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ced9a6a46a3544e9b4c186f02b721a19"
I20260812 01:13:02.067055 14274 raft_consensus.cc:3037] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:02.067811 14044 raft_consensus.cc:2454] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4b5066de6b1476088ea14c638be6864 in term 1.
I20260812 01:13:02.067817 14274 raft_consensus.cc:2454] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4b5066de6b1476088ea14c638be6864 in term 1.
I20260812 01:13:02.068013 13678 leader_election.cc:302] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ced9a6a46a3544e9b4c186f02b721a19, e4b5066de6b1476088ea14c638be6864; no voters:
I20260812 01:13:02.068147 14399 raft_consensus.cc:2781] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:02.068333 14399 raft_consensus.cc:686] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 1 LEADER]: Becoming Leader. State: Replica: e4b5066de6b1476088ea14c638be6864, State: Running, Role: LEADER
I20260812 01:13:02.068477 14399 consensus_queue.cc:237] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } }
I20260812 01:13:02.069651 13535 catalog_manager.cc:5495] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4b5066de6b1476088ea14c638be6864 (127.17.130.1). New cstate: current_term: 1 leader_uuid: "e4b5066de6b1476088ea14c638be6864" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4b5066de6b1476088ea14c638be6864" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34511 } health_report { overall_health: HEALTHY } } }
I20260812 01:13:02.102120 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.099522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.17.130.1:34511)
0812 01:13:02.099638 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.099641 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.099652 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:13:02.099734 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.099735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.099757 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.099844 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:02.099989 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.100480 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.101295 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.101489 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.101563 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.101736 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:02.101856 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:02.101950 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.101996 (+ 46us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.102018 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260812 01:13:02.104312 14274 raft_consensus.cc:1261] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 1 FOLLOWER]: Refusing update from remote peer e4b5066de6b1476088ea14c638be6864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:02.104346 14044 raft_consensus.cc:1261] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 1 FOLLOWER]: Refusing update from remote peer e4b5066de6b1476088ea14c638be6864: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:02.104743 14407 consensus_queue.cc:1035] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9f11676964d4beeaa1c12e18c18bad8" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 38071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:02.104892 14399 consensus_queue.cc:1035] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ced9a6a46a3544e9b4c186f02b721a19" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:02.108289 14412 mvcc.cc:205] Tried to move back new op lower bound from 7317492457895923712 to 7317492457753235456. Current Snapshot: MvccSnapshot[applied={T|T < 7317492457895923712}]
I20260812 01:13:02.108646 14413 mvcc.cc:205] Tried to move back new op lower bound from 7317492457895923712 to 7317492457753235456. Current Snapshot: MvccSnapshot[applied={T|T < 7317492457895923712 or (T in {7317492457895923712})}]
I20260812 01:13:02.110440 13535 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:02.110639 13535 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45402:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:02.111671 13535 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7b5118f6eb5456b9323992e5413c398: Sending DeleteTablet for 3 replicas of tablet a69b384287b84b27a844f6da83a069a8
I20260812 01:13:02.112152 14013 tablet_service.cc:1514] Processing DeleteTablet for tablet a69b384287b84b27a844f6da83a069a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:02 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:60056
I20260812 01:13:02.112211 14249 tablet_service.cc:1514] Processing DeleteTablet for tablet a69b384287b84b27a844f6da83a069a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:02 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:58272
I20260812 01:13:02.112273 13754 tablet_service.cc:1514] Processing DeleteTablet for tablet a69b384287b84b27a844f6da83a069a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:02 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:50006
I20260812 01:13:02.112500 14435 tablet_replica.cc:330] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: stopping tablet replica
I20260812 01:13:02.112495 14430 tablet_replica.cc:330] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: stopping tablet replica
I20260812 01:13:02.112620 14435 raft_consensus.cc:2229] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:13:02.112747 14435 raft_consensus.cc:2258] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:02.112771 14430 raft_consensus.cc:2229] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:02.112835 14431 tablet_replica.cc:330] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: stopping tablet replica
I20260812 01:13:02.112944 14431 raft_consensus.cc:2229] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:02.113121 14431 raft_consensus.cc:2258] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:02.113132 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 13642
I20260812 01:13:02.113406 14435 ts_tablet_manager.cc:1905] T a69b384287b84b27a844f6da83a069a8 P e4b5066de6b1476088ea14c638be6864: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:02.114120 14431 ts_tablet_manager.cc:1905] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:02.116446 14431 ts_tablet_manager.cc:1918] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:02.116518 14431 log.cc:1196] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/a69b384287b84b27a844f6da83a069a8
I20260812 01:13:02.116762 14431 ts_tablet_manager.cc:1939] T a69b384287b84b27a844f6da83a069a8 P e9f11676964d4beeaa1c12e18c18bad8: Deleting consensus metadata
I20260812 01:13:02.118232 13505 catalog_manager.cc:4841] TS e9f11676964d4beeaa1c12e18c18bad8 (127.17.130.2:38071): tablet a69b384287b84b27a844f6da83a069a8 (table test-table [id=c7010cad6f4a48eb8d34e69bb60c33a8]) successfully deleted
I20260812 01:13:02.113132 14430 raft_consensus.cc:2258] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:02.151273 14430 ts_tablet_manager.cc:1905] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:02.153542 14430 ts_tablet_manager.cc:1918] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:02.153661 14430 log.cc:1196] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/a69b384287b84b27a844f6da83a069a8
I20260812 01:13:02.154002 14430 ts_tablet_manager.cc:1939] T a69b384287b84b27a844f6da83a069a8 P ced9a6a46a3544e9b4c186f02b721a19: Deleting consensus metadata
I20260812 01:13:02.154608 13505 catalog_manager.cc:4841] TS ced9a6a46a3544e9b4c186f02b721a19 (127.17.130.3:36751): tablet a69b384287b84b27a844f6da83a069a8 (table test-table [id=c7010cad6f4a48eb8d34e69bb60c33a8]) successfully deleted
W20260812 01:13:02.156571 13502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:34511 (error 108)
I20260812 01:13:02.156942 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 13889
I20260812 01:13:02.157305 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.157176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34511 (local address 127.0.0.1:50026)
0812 01:13:02.157230 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.157264 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:13:02.157426 13502 catalog_manager.cc:4584] TS e4b5066de6b1476088ea14c638be6864 (127.17.130.1:34511): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a69b384287b84b27a844f6da83a069a8: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34511: connect: Connection refused (error 111)
I20260812 01:13:02.176342 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 14136
I20260812 01:13:02.193842 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 13470
2026-08-12T01:13:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2819 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T01:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:02Z Disabled control of system clock
I20260812 01:13:02.237246 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:39341
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:39341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:02.334498 14446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:02.334750 14446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:02.334774 14446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:02.335913 14446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:02.336051 14446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:02.336067 14446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:02.336082 14446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:02.336095 14446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:02.338303 14446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:39341
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:39341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:02.338649 14446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:02.339162 14446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:02.345979 14461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.345988 14454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.346000 14455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:02.346421 14446 server_base.cc:1034] running on GCE node
I20260812 01:13:02.346722 14446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:02.347056 14446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:02.348276 14446 hybrid_clock.cc:648] HybridClock initialized: now 1786497182348260 us; error 33 us; skew 500 ppm
I20260812 01:13:02.349774 14446 webserver.cc:460] Webserver started at http://127.17.130.62:38193/ using document root <none> and password file <none>
I20260812 01:13:02.350051 14446 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:02.350126 14446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:02.350279 14446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:02.351408 14446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "4f674cbccae84c11885962e44ed2dd26"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "f5096bc59fa13a18a2faedfe43cfcdf2"
server_key_iv: "abd198fd3e531e6adef2b8a0c70b1935"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.351786 14446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "4f674cbccae84c11885962e44ed2dd26"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "f5096bc59fa13a18a2faedfe43cfcdf2"
server_key_iv: "abd198fd3e531e6adef2b8a0c70b1935"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.353492 14446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:02.354380 14471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.354578 14446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:02.354689 14446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "4f674cbccae84c11885962e44ed2dd26"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "f5096bc59fa13a18a2faedfe43cfcdf2"
server_key_iv: "abd198fd3e531e6adef2b8a0c70b1935"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.354768 14446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:02.367337 14446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:02.367727 14446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:02.367908 14446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:02.372140 14446 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:39341
I20260812 01:13:02.372215 14580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:39341 every 8 connection(s)
I20260812 01:13:02.372598 14446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:02.372952 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 14446
I20260812 01:13:02.373086 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:02.373281 17928 external_mini_cluster.cc:1428] Setting key df2341efb58b103288d0c7d469e5e7d8
I20260812 01:13:02.374003 14584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:02.377163 14584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26: Bootstrap starting.
I20260812 01:13:02.378012 14584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:02.378384 14584 log.cc:824] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:02.379316 14584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26: No bootstrap required, opened a new log
I20260812 01:13:02.380059 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.373887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.17.130.62:39341)
0812 01:13:02.374107 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.374123 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.374157 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:13:02.374612 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.374620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.374682 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.374802 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:02.375211 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.376480 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.377693 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.378298 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.378400 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.379208 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:02.379212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:02.379213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:02.379414 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:02.379456 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:02.379469 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:02.379522 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.379628 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.379694 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260812 01:13:02.381583 14584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f674cbccae84c11885962e44ed2dd26" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39341 } }
I20260812 01:13:02.381740 14584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:02.381765 14584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f674cbccae84c11885962e44ed2dd26, State: Initialized, Role: FOLLOWER
I20260812 01:13:02.381897 14584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f674cbccae84c11885962e44ed2dd26" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39341 } }
I20260812 01:13:02.381975 14584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:02.381999 14584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:02.382028 14584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:02.382762 14584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f674cbccae84c11885962e44ed2dd26" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39341 } }
I20260812 01:13:02.382864 14584 leader_election.cc:302] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f674cbccae84c11885962e44ed2dd26; no voters:
I20260812 01:13:02.383092 14584 leader_election.cc:288] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:02.383138 14592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:02.383330 14592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [term 1 LEADER]: Becoming Leader. State: Replica: 4f674cbccae84c11885962e44ed2dd26, State: Running, Role: LEADER
I20260812 01:13:02.383420 14584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:02.383426 14592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f674cbccae84c11885962e44ed2dd26" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39341 } }
I20260812 01:13:02.383790 14600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f674cbccae84c11885962e44ed2dd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f674cbccae84c11885962e44ed2dd26" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39341 } } }
I20260812 01:13:02.383774 14592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f674cbccae84c11885962e44ed2dd26. Latest consensus state: current_term: 1 leader_uuid: "4f674cbccae84c11885962e44ed2dd26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f674cbccae84c11885962e44ed2dd26" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39341 } } }
I20260812 01:13:02.383883 14600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:02.383889 14592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:02.384219 14606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:02.384820 14606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:02.386652 14606 catalog_manager.cc:1347] Generated new cluster ID: cec9c27f377c4055a3c216fb94644d09
I20260812 01:13:02.386715 14606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:02.406968 14606 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:02.407719 14606 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:02.418481 14606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26: Generated new TSK 0
I20260812 01:13:02.418818 14606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:02.422693 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39341
--builtin_ntp_servers=127.17.130.20:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:02.526819 14623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:02.527141 14623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:02.527186 14623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:02.528707 14623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:02.528789 14623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:02.528842 14623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:13:02.530609 14623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:02.530927 14623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:02.531291 14623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:02.538280 14635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.538360 14634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.538364 14640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:02.538745 14623 server_base.cc:1034] running on GCE node
I20260812 01:13:02.538970 14623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:02.539279 14623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:02.540457 14623 hybrid_clock.cc:648] HybridClock initialized: now 1786497182540444 us; error 40 us; skew 500 ppm
I20260812 01:13:02.542075 14623 webserver.cc:460] Webserver started at http://127.17.130.1:46055/ using document root <none> and password file <none>
I20260812 01:13:02.542423 14623 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:02.542495 14623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:02.542970 14623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:02.544812 14623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "d57b2003189a42e8a83b0d2a2c26922b"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "348ace653dfcd9438dac6ed344ca7398"
server_key_iv: "0f6c8f20ddf1640f33ff0b8454edc60b"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.545257 14623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "d57b2003189a42e8a83b0d2a2c26922b"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "348ace653dfcd9438dac6ed344ca7398"
server_key_iv: "0f6c8f20ddf1640f33ff0b8454edc60b"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.547248 14623 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 01:13:02.548322 14650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.548630 14623 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:02.548733 14623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "d57b2003189a42e8a83b0d2a2c26922b"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "348ace653dfcd9438dac6ed344ca7398"
server_key_iv: "0f6c8f20ddf1640f33ff0b8454edc60b"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.548812 14623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:02.563676 14623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:02.564009 14623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:02.564327 14623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:02.564581 14623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:02.564898 14623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:02.564945 14623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.564970 14623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:02.564982 14623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.572333 14623 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:36049
I20260812 01:13:02.572381 14847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:36049 every 8 connection(s)
I20260812 01:13:02.572760 14623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:13:02.579190 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.573625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:44773 (local address 127.17.130.62:39341)
0812 01:13:02.573705 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.573708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.573928 (+ 220us) server_negotiation.cc:407] Connection header received
0812 01:13:02.574722 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.574723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.574743 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.574806 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:02.575999 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.576533 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.577658 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.577859 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.577936 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.578863 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:02.578865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:02.578866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:02.578872 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:02.578888 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:02.578892 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:02.578958 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.579057 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.579097 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 01:13:02.579449 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.573603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39341 (local address 127.17.130.1:44773)
0812 01:13:02.573806 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.573841 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.574524 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.575173 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.575182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.575193 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:02.575797 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.575802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.576641 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.576642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.577559 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.577562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.577946 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.577947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.578093 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.578706 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:02.578707 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:02.578738 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:02.578764 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:02.578962 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:02.578969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:02.578972 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.579058 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.579184 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260812 01:13:02.580150 14848 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39341
I20260812 01:13:02.580222 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 14623
I20260812 01:13:02.580291 14848 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:02.580310 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:13:02.580511 17928 external_mini_cluster.cc:1428] Setting key 1ea0e44f17d6f369a78644f96ee059b2
I20260812 01:13:02.580503 14848 heartbeater.cc:511] Master 127.17.130.62:39341 requested a full tablet report, sending...
I20260812 01:13:02.581142 14499 ts_manager.cc:194] Registered new tserver with Master: d57b2003189a42e8a83b0d2a2c26922b (127.17.130.1:36049)
I20260812 01:13:02.581704 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39341
--builtin_ntp_servers=127.17.130.20:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:13:02.582026 14499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.1:44773
W20260812 01:13:02.676476 14851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:02.676719 14851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:02.676744 14851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:02.677891 14851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:02.677933 14851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:02.677965 14851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:13:02.679711 14851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:02.680064 14851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:02.680411 14851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:02.686149 14863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.686178 14862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.686164 14867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:02.686210 14851 server_base.cc:1034] running on GCE node
I20260812 01:13:02.686668 14851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:02.686957 14851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:02.688123 14851 hybrid_clock.cc:648] HybridClock initialized: now 1786497182688097 us; error 39 us; skew 500 ppm
I20260812 01:13:02.689532 14851 webserver.cc:460] Webserver started at http://127.17.130.2:35995/ using document root <none> and password file <none>
I20260812 01:13:02.689805 14851 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:02.689877 14851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:02.690032 14851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:02.691146 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "ee160ae0a60743279a7f8bba58b585ba"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "e1d1f4fef2598f35f1cbce33fd94e68b"
server_key_iv: "49d03b2a844d96441703d70c1fb98cb9"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.691531 14851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "ee160ae0a60743279a7f8bba58b585ba"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "e1d1f4fef2598f35f1cbce33fd94e68b"
server_key_iv: "49d03b2a844d96441703d70c1fb98cb9"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.693044 14851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:13:02.694273 14879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.694458 14851 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:02.694582 14851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "ee160ae0a60743279a7f8bba58b585ba"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "e1d1f4fef2598f35f1cbce33fd94e68b"
server_key_iv: "49d03b2a844d96441703d70c1fb98cb9"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.694659 14851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:02.710456 14851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:02.710816 14851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:02.710978 14851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:02.711228 14851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:02.711572 14851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:02.711632 14851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.711674 14851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:02.711707 14851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.718209 14851 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:34653
I20260812 01:13:02.718264 15078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:34653 every 8 connection(s)
I20260812 01:13:02.718649 14851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:13:02.719012 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 14851
I20260812 01:13:02.719131 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:13:02.719375 17928 external_mini_cluster.cc:1428] Setting key cbfbded4d873a51fdbe1e419d7becca1
I20260812 01:13:02.720423 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39341
--builtin_ntp_servers=127.17.130.20:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:13:02.724841 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.719540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:38531 (local address 127.17.130.62:39341)
0812 01:13:02.719591 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.719594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.719810 (+ 216us) server_negotiation.cc:407] Connection header received
0812 01:13:02.720547 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.720549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.720563 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.720655 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:02.721630 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.722133 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.723207 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.723510 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.723587 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.724573 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:02.724574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:02.724574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:02.724580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:02.724595 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:02.724600 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:02.724652 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.724728 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.724757 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:13:02.725124 15083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.719468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39341 (local address 127.17.130.2:38531)
0812 01:13:02.719683 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.719719 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.720377 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.720727 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.720731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.720742 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:02.721501 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.721508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.722241 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.722242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.723119 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.723121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.723623 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.723624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.723761 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.724397 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:02.724398 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:02.724427 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:02.724451 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:02.724658 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:02.724664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:02.724667 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.724734 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.724842 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260812 01:13:02.725690 15080 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39341
I20260812 01:13:02.725813 15080 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:02.726045 15080 heartbeater.cc:511] Master 127.17.130.62:39341 requested a full tablet report, sending...
I20260812 01:13:02.726501 14499 ts_manager.cc:194] Registered new tserver with Master: ee160ae0a60743279a7f8bba58b585ba (127.17.130.2:34653)
I20260812 01:13:02.727072 14499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.2:38531
W20260812 01:13:02.817674 15084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:02.818005 15084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:02.818037 15084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:02.819667 15084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:02.819720 15084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:02.819751 15084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:13:02.821472 15084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39341
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:02.821794 15084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:02.822431 15084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:02.828923 15092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.828943 15094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:02.828904 15096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:02.829232 15084 server_base.cc:1034] running on GCE node
I20260812 01:13:02.829463 15084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:02.829735 15084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:02.830926 15084 hybrid_clock.cc:648] HybridClock initialized: now 1786497182830915 us; error 36 us; skew 500 ppm
I20260812 01:13:02.832530 15084 webserver.cc:460] Webserver started at http://127.17.130.3:42225/ using document root <none> and password file <none>
I20260812 01:13:02.832854 15084 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:02.832931 15084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:02.833133 15084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:02.834501 15084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "b7a74eba9272486fa7e08ef6f00b0983"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "f488447210ce98eaf0e214b939b2bf0d"
server_key_iv: "62507f79a1a6988c264eb13fb0c120ae"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.835330 15084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "b7a74eba9272486fa7e08ef6f00b0983"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "f488447210ce98eaf0e214b939b2bf0d"
server_key_iv: "62507f79a1a6988c264eb13fb0c120ae"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.837337 15084 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:13:02.838423 15104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.838758 15084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:02.838929 15084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "b7a74eba9272486fa7e08ef6f00b0983"
format_stamp: "Formatted at 2026-08-12 01:13:02 on dist-test-slave-tfpf"
server_key: "f488447210ce98eaf0e214b939b2bf0d"
server_key_iv: "62507f79a1a6988c264eb13fb0c120ae"
server_key_version: "encryptionkey@0"
I20260812 01:13:02.839020 15084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:02.865424 15084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:02.865800 15084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:02.865954 15084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:02.866216 15084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:02.866703 15084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:02.866760 15084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.866804 15084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:02.866832 15084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:02.873273 15084 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:36501
I20260812 01:13:02.873320 15310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:36501 every 8 connection(s)
I20260812 01:13:02.873685 15084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:13:02.878166 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 15084
I20260812 01:13:02.878297 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:13:02.878477 17928 external_mini_cluster.cc:1428] Setting key dea26e583ae4b2c0dac83e9313989527
I20260812 01:13:02.881630 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.874528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:35141 (local address 127.17.130.62:39341)
0812 01:13:02.874710 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.874713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.875366 (+ 653us) server_negotiation.cc:407] Connection header received
0812 01:13:02.876194 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.876196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.876208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.876264 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:02.877131 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.877650 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.879113 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.879496 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.879585 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.881244 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:02.881246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:02.881246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:02.881256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:02.881271 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:02.881276 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:02.881392 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.881516 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.881545 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:02.881878 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.874515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39341 (local address 127.17.130.3:35141)
0812 01:13:02.875193 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.875253 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.876065 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.876333 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.876336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.876345 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:02.876979 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.876985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.877756 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.877757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.878992 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.878995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.879612 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.879614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.879952 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.880959 (+ 1007us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:02.880959 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:02.881001 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:02.881047 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:02.881416 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:02.881423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:02.881427 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.881518 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.881639 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":247,"threads_started":1}
I20260812 01:13:02.882393 15311 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39341
I20260812 01:13:02.882486 15311 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:02.882686 15311 heartbeater.cc:511] Master 127.17.130.62:39341 requested a full tablet report, sending...
I20260812 01:13:02.883147 14499 ts_manager.cc:194] Registered new tserver with Master: b7a74eba9272486fa7e08ef6f00b0983 (127.17.130.3:36501)
I20260812 01:13:02.883699 14499 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.3:35141
I20260812 01:13:02.892731 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:13:02.897943 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.894343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.17.130.62:39341)
0812 01:13:02.894507 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.894510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.894520 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:02.894547 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.894548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.894565 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.894621 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:02.894951 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.895596 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.896391 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.896596 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.896658 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.897487 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:02.897489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:02.897490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:02.897496 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:02.897519 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:02.897524 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:02.897653 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.897843 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.897874 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260812 01:13:02.900647 14499 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:13:02.903373 14499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:13:02.910452 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.906273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.17.130.2:34653)
0812 01:13:02.906515 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.906520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.906735 (+ 215us) server_negotiation.cc:407] Connection header received
0812 01:13:02.906838 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.906840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.906857 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.906929 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:02.907190 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.908402 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.909536 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.910043 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.910129 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.910240 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.910317 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.910358 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":114,"threads_started":1}
I20260812 01:13:02.910439 15324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.906171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:34653 (local address 127.0.0.1:38872)
0812 01:13:02.906669 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.906676 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.906734 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.907021 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.907024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.907027 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:02.907108 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.907110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.908491 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.908492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.909443 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.909445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.910130 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.910132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.910209 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.910222 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.910301 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.910357 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260812 01:13:02.910797 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.906343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36049 (local address 127.0.0.1:40236)
0812 01:13:02.906690 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.906701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.906769 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.907086 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.907087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.907090 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:02.907171 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.907173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.908467 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.908468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.909582 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.909584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.910558 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.910559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.910633 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.910639 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.910689 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.910729 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":32,"thread_start_us":39,"threads_started":1}
I20260812 01:13:02.911058 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.906510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.17.130.1:36049)
0812 01:13:02.906681 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.906685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.906752 (+ 67us) server_negotiation.cc:407] Connection header received
0812 01:13:02.906840 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.906842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.906858 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.906938 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:02.907260 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.908375 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.909676 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.910391 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.910450 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.910526 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.910911 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.910959 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
I20260812 01:13:02.911384 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.906098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36501 (local address 127.0.0.1:38284)
0812 01:13:02.906457 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:02.906504 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:02.906570 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:02.907235 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:02.907236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:02.907238 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:02.907278 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.907279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.908149 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.908150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.909452 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:02.909455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.910491 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.910492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:02.911081 (+ 589us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.911087 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:13:02.911298 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 01:13:02.911337 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":84,"threads_started":1}
I20260812 01:13:02.911355 14970 tablet_service.cc:1467] Processing CreateTablet for tablet 00fec1ea1fe5434baa6793fae8aee435 (DEFAULT_TABLE table=test-table [id=de7ac3e6ec18444695929eb592e6e6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:02.911801 14970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00fec1ea1fe5434baa6793fae8aee435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:02.911751 14736 tablet_service.cc:1467] Processing CreateTablet for tablet 00fec1ea1fe5434baa6793fae8aee435 (DEFAULT_TABLE table=test-table [id=de7ac3e6ec18444695929eb592e6e6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:02.912089 14736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00fec1ea1fe5434baa6793fae8aee435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:02.913436 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:02.906291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38284 (local address 127.17.130.3:36501)
0812 01:13:02.906917 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:02.906921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:02.906931 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:02.906984 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:02.906986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:02.907063 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:02.907166 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:02.907335 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.908034 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.909551 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:02.910289 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:02.910363 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:02.910453 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:02.913273 (+ 2820us) server_negotiation.cc:299] Negotiation successful
0812 01:13:02.913328 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":121,"threads_started":1}
I20260812 01:13:02.914021 15199 tablet_service.cc:1467] Processing CreateTablet for tablet 00fec1ea1fe5434baa6793fae8aee435 (DEFAULT_TABLE table=test-table [id=de7ac3e6ec18444695929eb592e6e6eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:02.914367 15199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00fec1ea1fe5434baa6793fae8aee435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:02.915498 15336 tablet_bootstrap.cc:492] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Bootstrap starting.
I20260812 01:13:02.916350 15336 tablet_bootstrap.cc:654] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:02.916704 15336 log.cc:824] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:02.917143 15334 tablet_bootstrap.cc:492] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: Bootstrap starting.
I20260812 01:13:02.917583 15336 tablet_bootstrap.cc:492] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: No bootstrap required, opened a new log
I20260812 01:13:02.917672 15336 ts_tablet_manager.cc:1397] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:02.917899 15338 tablet_bootstrap.cc:492] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Bootstrap starting.
I20260812 01:13:02.917873 15334 tablet_bootstrap.cc:654] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:02.918334 15334 log.cc:824] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:02.918689 15338 tablet_bootstrap.cc:654] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:02.919221 15338 log.cc:824] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:02.919445 15334 tablet_bootstrap.cc:492] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: No bootstrap required, opened a new log
I20260812 01:13:02.919521 15334 ts_tablet_manager.cc:1397] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:13:02.920258 15338 tablet_bootstrap.cc:492] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: No bootstrap required, opened a new log
I20260812 01:13:02.920351 15338 ts_tablet_manager.cc:1397] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:13:02.921325 15336 raft_consensus.cc:348] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:02.921604 15336 raft_consensus.cc:374] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:02.921655 15336 raft_consensus.cc:729] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee160ae0a60743279a7f8bba58b585ba, State: Initialized, Role: FOLLOWER
I20260812 01:13:02.921792 15336 consensus_queue.cc:260] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:02.921720 15334 raft_consensus.cc:348] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:02.921926 15334 raft_consensus.cc:374] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:02.921969 15334 raft_consensus.cc:729] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d57b2003189a42e8a83b0d2a2c26922b, State: Initialized, Role: FOLLOWER
I20260812 01:13:02.922078 15334 consensus_queue.cc:260] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:02.922266 15336 ts_tablet_manager.cc:1428] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 01:13:02.922375 15334 ts_tablet_manager.cc:1428] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:02.922401 15080 heartbeater.cc:503] Master 127.17.130.62:39341 was elected leader, sending a full tablet report...
I20260812 01:13:02.922518 14848 heartbeater.cc:503] Master 127.17.130.62:39341 was elected leader, sending a full tablet report...
I20260812 01:13:02.923456 15338 raft_consensus.cc:348] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:02.923653 15338 raft_consensus.cc:374] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:02.923702 15338 raft_consensus.cc:729] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a74eba9272486fa7e08ef6f00b0983, State: Initialized, Role: FOLLOWER
I20260812 01:13:02.923821 15338 consensus_queue.cc:260] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:02.924167 15338 ts_tablet_manager.cc:1428] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260812 01:13:02.924204 15311 heartbeater.cc:503] Master 127.17.130.62:39341 was elected leader, sending a full tablet report...
W20260812 01:13:02.969321 15081 tablet.cc:2369] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:13:03.073633 14849 tablet.cc:2369] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:03.094775 15349 raft_consensus.cc:482] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:13:03.094965 15349 raft_consensus.cc:504] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:03.095366 15349 leader_election.cc:288] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee160ae0a60743279a7f8bba58b585ba (127.17.130.2:34653), d57b2003189a42e8a83b0d2a2c26922b (127.17.130.1:36049)
I20260812 01:13:03.098790 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:03.095796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:33869 (local address 127.17.130.1:36049)
0812 01:13:03.095878 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:03.095880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:03.095999 (+ 119us) server_negotiation.cc:407] Connection header received
0812 01:13:03.096084 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:03.096085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:03.096094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:03.096159 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:03.096315 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.096818 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.097944 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.098409 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.098496 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:03.098506 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:03.098663 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 01:13:03.098709 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 01:13:03.098806 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:03.095709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36049 (local address 127.17.130.3:33869)
0812 01:13:03.095900 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:03.095917 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:03.096015 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:03.096162 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:03.096162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:03.096164 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:03.096238 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:03.096240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.096912 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.096913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:03.097832 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:03.097834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.098496 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.098497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:03.098577 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:03.098581 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:03.098653 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:13:03.098706 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":117,"threads_started":1}
I20260812 01:13:03.099208 14774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00fec1ea1fe5434baa6793fae8aee435" candidate_uuid: "b7a74eba9272486fa7e08ef6f00b0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d57b2003189a42e8a83b0d2a2c26922b" is_pre_election: true
I20260812 01:13:03.099413 15313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:03.095600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:34653 (local address 127.17.130.3:34617)
0812 01:13:03.095678 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:03.095694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:03.095759 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:03.095953 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:03.095954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:03.095956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:03.096050 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:03.096052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.096995 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.096997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:03.097849 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:03.097852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.099168 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.099169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:03.099232 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:03.099235 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:03.099280 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:13:03.099316 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 01:13:03.099418 14774 raft_consensus.cc:2454] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7a74eba9272486fa7e08ef6f00b0983 in term 0.
I20260812 01:13:03.099473 15331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:03.095755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:34617 (local address 127.17.130.2:34653)
0812 01:13:03.095812 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:03.095814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:03.095822 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:13:03.095854 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:03.095855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:03.095867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:03.095944 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:03.096191 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.096886 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.098166 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.098965 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.099111 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:03.099130 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:03.099319 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 01:13:03.099397 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 01:13:03.099691 15112 leader_election.cc:302] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7a74eba9272486fa7e08ef6f00b0983, d57b2003189a42e8a83b0d2a2c26922b; no voters:
I20260812 01:13:03.099740 15008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00fec1ea1fe5434baa6793fae8aee435" candidate_uuid: "b7a74eba9272486fa7e08ef6f00b0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee160ae0a60743279a7f8bba58b585ba" is_pre_election: true
I20260812 01:13:03.099848 15349 raft_consensus.cc:2781] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:13:03.099915 15349 raft_consensus.cc:482] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:13:03.099905 15008 raft_consensus.cc:2454] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7a74eba9272486fa7e08ef6f00b0983 in term 0.
I20260812 01:13:03.099949 15349 raft_consensus.cc:3037] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:03.100759 15349 raft_consensus.cc:504] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:03.100992 15349 leader_election.cc:288] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [CANDIDATE]: Term 1 election: Requested vote from peers ee160ae0a60743279a7f8bba58b585ba (127.17.130.2:34653), d57b2003189a42e8a83b0d2a2c26922b (127.17.130.1:36049)
I20260812 01:13:03.101087 15008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00fec1ea1fe5434baa6793fae8aee435" candidate_uuid: "b7a74eba9272486fa7e08ef6f00b0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee160ae0a60743279a7f8bba58b585ba"
I20260812 01:13:03.101214 15008 raft_consensus.cc:3037] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:03.101207 14774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00fec1ea1fe5434baa6793fae8aee435" candidate_uuid: "b7a74eba9272486fa7e08ef6f00b0983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d57b2003189a42e8a83b0d2a2c26922b"
I20260812 01:13:03.101425 14774 raft_consensus.cc:3037] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:03.102243 15008 raft_consensus.cc:2454] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7a74eba9272486fa7e08ef6f00b0983 in term 1.
I20260812 01:13:03.102399 14774 raft_consensus.cc:2454] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7a74eba9272486fa7e08ef6f00b0983 in term 1.
I20260812 01:13:03.102432 15112 leader_election.cc:302] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7a74eba9272486fa7e08ef6f00b0983, ee160ae0a60743279a7f8bba58b585ba; no voters:
I20260812 01:13:03.102537 15349 raft_consensus.cc:2781] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:03.102726 15349 raft_consensus.cc:686] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 1 LEADER]: Becoming Leader. State: Replica: b7a74eba9272486fa7e08ef6f00b0983, State: Running, Role: LEADER
I20260812 01:13:03.102866 15349 consensus_queue.cc:237] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } }
I20260812 01:13:03.103878 14499 catalog_manager.cc:5495] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7a74eba9272486fa7e08ef6f00b0983 (127.17.130.3). New cstate: current_term: 1 leader_uuid: "b7a74eba9272486fa7e08ef6f00b0983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7a74eba9272486fa7e08ef6f00b0983" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 } health_report { overall_health: UNKNOWN } } }
W20260812 01:13:03.124421 15312 tablet.cc:2369] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:03.144627 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:03.142000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38292 (local address 127.17.130.3:36501)
0812 01:13:03.142080 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:03.142082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:03.142098 (+ 16us) server_negotiation.cc:407] Connection header received
0812 01:13:03.142148 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:03.142148 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:03.142172 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:03.142245 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:03.142382 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.142927 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.143763 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:03.143985 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:03.144066 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:03.144251 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:03.144367 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:03.144465 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:03.144484 (+ 19us) server_negotiation.cc:299] Negotiation successful
0812 01:13:03.144505 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:13:03.146728 15008 raft_consensus.cc:1261] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 1 FOLLOWER]: Refusing update from remote peer b7a74eba9272486fa7e08ef6f00b0983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:03.146728 14774 raft_consensus.cc:1261] T 00fec1ea1fe5434baa6793fae8aee435 P d57b2003189a42e8a83b0d2a2c26922b [term 1 FOLLOWER]: Refusing update from remote peer b7a74eba9272486fa7e08ef6f00b0983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:03.147117 15349 consensus_queue.cc:1035] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d57b2003189a42e8a83b0d2a2c26922b" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:03.147277 15351 consensus_queue.cc:1035] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee160ae0a60743279a7f8bba58b585ba" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 34653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:03.150756 15363 mvcc.cc:205] Tried to move back new op lower bound from 7317492462166003712 to 7317492461990207488. Current Snapshot: MvccSnapshot[applied={T|T < 7317492462166003712 or (T in {7317492462166003712})}]
I20260812 01:13:03.152429 14497 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:03.152758 14497 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:03.153733 14497 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f674cbccae84c11885962e44ed2dd26: Sending DeleteTablet for 3 replicas of tablet 00fec1ea1fe5434baa6793fae8aee435
I20260812 01:13:03.154100 15199 tablet_service.cc:1514] Processing DeleteTablet for tablet 00fec1ea1fe5434baa6793fae8aee435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:03 UTC) from {username='slave'} at 127.0.0.1:38284
I20260812 01:13:03.154141 14970 tablet_service.cc:1514] Processing DeleteTablet for tablet 00fec1ea1fe5434baa6793fae8aee435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:03 UTC) from {username='slave'} at 127.0.0.1:38872
I20260812 01:13:03.154448 14736 tablet_service.cc:1514] Processing DeleteTablet for tablet 00fec1ea1fe5434baa6793fae8aee435 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:03 UTC) from {username='slave'} at 127.0.0.1:40236
I20260812 01:13:03.155092 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 14623
I20260812 01:13:03.164002 15377 tablet_replica.cc:330] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: stopping tablet replica
I20260812 01:13:03.164197 15377 raft_consensus.cc:2229] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:03.164335 15377 raft_consensus.cc:2258] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:03.165601 15377 ts_tablet_manager.cc:1905] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:03.167582 15377 ts_tablet_manager.cc:1918] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:03.167757 15377 log.cc:1196] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/00fec1ea1fe5434baa6793fae8aee435
I20260812 01:13:03.168074 15377 ts_tablet_manager.cc:1939] T 00fec1ea1fe5434baa6793fae8aee435 P ee160ae0a60743279a7f8bba58b585ba: Deleting consensus metadata
I20260812 01:13:03.168697 15376 tablet_replica.cc:330] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: stopping tablet replica
I20260812 01:13:03.168766 14476 catalog_manager.cc:4841] TS ee160ae0a60743279a7f8bba58b585ba (127.17.130.2:34653): tablet 00fec1ea1fe5434baa6793fae8aee435 (table test-table [id=de7ac3e6ec18444695929eb592e6e6eb]) successfully deleted
I20260812 01:13:03.168856 15376 raft_consensus.cc:2229] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:13:03.169011 15376 raft_consensus.cc:2258] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:03.169476 15376 ts_tablet_manager.cc:1905] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:03.170832 15376 ts_tablet_manager.cc:1918] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:03.170923 15376 log.cc:1196] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/00fec1ea1fe5434baa6793fae8aee435
I20260812 01:13:03.171171 15376 ts_tablet_manager.cc:1939] T 00fec1ea1fe5434baa6793fae8aee435 P b7a74eba9272486fa7e08ef6f00b0983: Deleting consensus metadata
I20260812 01:13:03.171644 14474 catalog_manager.cc:4841] TS b7a74eba9272486fa7e08ef6f00b0983 (127.17.130.3:36501): tablet 00fec1ea1fe5434baa6793fae8aee435 (table test-table [id=de7ac3e6ec18444695929eb592e6e6eb]) successfully deleted
W20260812 01:13:03.176337 14476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:36049 (error 108)
I20260812 01:13:03.176972 15326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:03.176823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:36049 (local address 127.0.0.1:40250)
0812 01:13:03.176889 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:03.176936 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 01:13:03.177120 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 14851
W20260812 01:13:03.189924 14476 catalog_manager.cc:4584] TS d57b2003189a42e8a83b0d2a2c26922b (127.17.130.1:36049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 00fec1ea1fe5434baa6793fae8aee435: Network error: Client connection negotiation failed: client connection to 127.17.130.1:36049: connect: Connection refused (error 111)
I20260812 01:13:03.196954 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 15084
I20260812 01:13:03.209131 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 14446
2026-08-12T01:13:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1007 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:03 dist-test-slave-tfpf krb5kdc[15386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:03 dist-test-slave-tfpf krb5kdc[15386](info): set up 2 sockets
Aug 12 01:13:03 dist-test-slave-tfpf krb5kdc[15386](info): commencing operation
krb5kdc: starting...
W20260812 01:13:04.590102 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:04 dist-test-slave-tfpf krb5kdc[15386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:04.745749 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:42787
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:42787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:04.842702 15410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:04.843374 15410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:04.843433 15410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:04.845551 15410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:04.845654 15410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:04.845679 15410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:04.845703 15410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:04.845727 15410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:04.847954 15410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:42787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:42787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:04.848358 15410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:04.848676 15410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:04.855912 15419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:04.855916 15416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:04.855914 15417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:04.857038 15410 server_base.cc:1034] running on GCE node
I20260812 01:13:04.857450 15410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:04.857894 15410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:04.859133 15410 hybrid_clock.cc:648] HybridClock initialized: now 1786497184859086 us; error 99 us; skew 500 ppm
Aug 12 01:13:04 dist-test-slave-tfpf krb5kdc[15386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497184, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:04.867792 15410 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:04.868423 15410 webserver.cc:460] Webserver started at http://127.17.130.62:34823/ using document root <none> and password file <none>
I20260812 01:13:04.868683 15410 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:04.868752 15410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:04.868906 15410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:04.870152 15410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "80fe137164614924986e3c2ccbd9190c"
format_stamp: "Formatted at 2026-08-12 01:13:04 on dist-test-slave-tfpf"
server_key: "3b563a893e7ada92b033adae3f46f4b9"
server_key_iv: "1cdbe0782c97c927655d2dd745bb381d"
server_key_version: "encryptionkey@0"
I20260812 01:13:04.870568 15410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "80fe137164614924986e3c2ccbd9190c"
format_stamp: "Formatted at 2026-08-12 01:13:04 on dist-test-slave-tfpf"
server_key: "3b563a893e7ada92b033adae3f46f4b9"
server_key_iv: "1cdbe0782c97c927655d2dd745bb381d"
server_key_version: "encryptionkey@0"
I20260812 01:13:04.872145 15410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:13:04.873231 15432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:04.873574 15410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:04.873701 15410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "80fe137164614924986e3c2ccbd9190c"
format_stamp: "Formatted at 2026-08-12 01:13:04 on dist-test-slave-tfpf"
server_key: "3b563a893e7ada92b033adae3f46f4b9"
server_key_iv: "1cdbe0782c97c927655d2dd745bb381d"
server_key_version: "encryptionkey@0"
I20260812 01:13:04.873780 15410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:04.896994 15410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:04.900564 15410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:04.900775 15410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:04.906270 15410 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:42787
I20260812 01:13:04.906383 15526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:42787 every 8 connection(s)
I20260812 01:13:04.906793 15410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:04.907832 15527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:04.910457 15527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c: Bootstrap starting.
I20260812 01:13:04.911187 15527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:04.911521 15527 log.cc:824] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:04.912533 15527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c: No bootstrap required, opened a new log
I20260812 01:13:04.913661 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 15410
I20260812 01:13:04.913763 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:04.913968 17928 external_mini_cluster.cc:1428] Setting key 117c10a31450f0b89a198784156cde93
I20260812 01:13:04.914880 15527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fe137164614924986e3c2ccbd9190c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42787 } }
I20260812 01:13:04.915064 15527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:04.915112 15527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fe137164614924986e3c2ccbd9190c, State: Initialized, Role: FOLLOWER
I20260812 01:13:04.915254 15527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fe137164614924986e3c2ccbd9190c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42787 } }
I20260812 01:13:04.915350 15527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:04.915396 15527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:04.915459 15527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:04.916591 15527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fe137164614924986e3c2ccbd9190c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42787 } }
I20260812 01:13:04.916757 15527 leader_election.cc:302] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80fe137164614924986e3c2ccbd9190c; no voters:
I20260812 01:13:04.917004 15527 leader_election.cc:288] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:04.917024 15540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:04.917245 15540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [term 1 LEADER]: Becoming Leader. State: Replica: 80fe137164614924986e3c2ccbd9190c, State: Running, Role: LEADER
I20260812 01:13:04.917369 15527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:04.917356 15540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fe137164614924986e3c2ccbd9190c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42787 } }
I20260812 01:13:04.917990 15542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80fe137164614924986e3c2ccbd9190c. Latest consensus state: current_term: 1 leader_uuid: "80fe137164614924986e3c2ccbd9190c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fe137164614924986e3c2ccbd9190c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42787 } } }
I20260812 01:13:04.917996 15541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80fe137164614924986e3c2ccbd9190c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80fe137164614924986e3c2ccbd9190c" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 42787 } } }
I20260812 01:13:04.918123 15542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:04.918128 15541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:04.918568 15553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:04.919230 15553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:04.920974 15553 catalog_manager.cc:1347] Generated new cluster ID: 80e7666b0b484f7b8eb235eeead0aaa1
I20260812 01:13:04.921034 15553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:13:04 dist-test-slave-tfpf krb5kdc[15386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:04.926712 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:04.914623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.17.130.62:42787)
0812 01:13:04.914838 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:04.914847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:04.914875 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:13:04.915007 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:04.915013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:04.915057 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:04.915219 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:04.916104 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:04.917910 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:04.919081 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:04.919618 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:04.919756 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:04.922512 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:04.922517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:04.922518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:04.922543 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:04.924771 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:04.925329 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:04.925337 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:04.925338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:04.925452 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:04.925670 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:04.925671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:04.925671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:04.925943 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:04.926232 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:04.926347 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 01:13:04.926513 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":120,"threads_started":1}
I20260812 01:13:04.939536 15553 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:04.940375 15553 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:04.947994 15553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c: Generated new TSK 0
I20260812 01:13:04.948251 15553 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:13:04.992300 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:42787
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:05.098713 15569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:05.099092 15569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:05.099298 15569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:05.102170 15569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:05.102243 15569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:05.102278 15569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:13:05.104419 15569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:42787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:05.104751 15569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:05.105049 15569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:05.111310 15575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:05.111332 15569 server_base.cc:1034] running on GCE node
W20260812 01:13:05.111382 15580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.111361 15577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:05.111810 15569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:05.112142 15569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:05.113315 15569 hybrid_clock.cc:648] HybridClock initialized: now 1786497185113283 us; error 55 us; skew 500 ppm
Aug 12 01:13:05 dist-test-slave-tfpf krb5kdc[15386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:05.116227 15569 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:13:05.116755 15569 webserver.cc:460] Webserver started at http://127.17.130.1:32829/ using document root <none> and password file <none>
I20260812 01:13:05.117002 15569 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:05.117045 15569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:05.117233 15569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:05.118772 15569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "c745bb4c135947708d96adafffc6637d"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "7a7cbcab833f507abe39a798741759b8"
server_key_iv: "60700519859eb11a9f3f1ebe90d00bda"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.119103 15569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "c745bb4c135947708d96adafffc6637d"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "7a7cbcab833f507abe39a798741759b8"
server_key_iv: "60700519859eb11a9f3f1ebe90d00bda"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.120517 15569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:13:05.121377 15592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.121659 15569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:05.121762 15569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "c745bb4c135947708d96adafffc6637d"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "7a7cbcab833f507abe39a798741759b8"
server_key_iv: "60700519859eb11a9f3f1ebe90d00bda"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.121824 15569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:05.132995 15569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:05.136106 15569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:05.136255 15569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:05.136488 15569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:05.137214 15569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:05.137282 15569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.137310 15569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:05.137322 15569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.144639 15569 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:38529
I20260812 01:13:05.144832 15792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:38529 every 8 connection(s)
I20260812 01:13:05.145188 15569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:13:05.152376 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 15569
Aug 12 01:13:05 dist-test-slave-tfpf krb5kdc[15386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:05.152542 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:13:05.152786 17928 external_mini_cluster.cc:1428] Setting key 50569681a9157a5094138db25e3d7392
I20260812 01:13:05.157378 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.146327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:42987 (local address 127.17.130.62:42787)
0812 01:13:05.146378 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.146380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.146581 (+ 201us) server_negotiation.cc:407] Connection header received
0812 01:13:05.147085 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.147087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.147103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.147181 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:05.148258 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.148790 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.149962 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.150200 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.150272 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.153428 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:05.153430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:05.153430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:05.153442 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:05.155319 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.155942 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.155943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.155944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.156031 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.156375 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.156376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.156376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.156628 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:05.156746 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.157106 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.157243 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:05.157509 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.146247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:42787 (local address 127.17.130.1:42987)
0812 01:13:05.146428 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.146474 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.146968 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.147266 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.147269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.147904 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:05.148143 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.148148 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.148899 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.148900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.149846 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.149849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.150308 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.150309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.150434 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.151033 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:05.151042 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:05.153197 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:05.155425 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:05.155437 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:05.155440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:05.155851 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:05.156135 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:05.156136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:05.156137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:05.156294 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:05.156759 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:05.156766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:05.156927 (+ 161us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.157038 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.157280 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260812 01:13:05.158510 15795 heartbeater.cc:347] Connected to a master server at 127.17.130.62:42787
I20260812 01:13:05.158748 15795 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:05.159135 15795 heartbeater.cc:511] Master 127.17.130.62:42787 requested a full tablet report, sending...
I20260812 01:13:05.159880 15457 ts_manager.cc:194] Registered new tserver with Master: c745bb4c135947708d96adafffc6637d (127.17.130.1:38529)
I20260812 01:13:05.160902 15457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:42987
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:13:05.185112 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:42787
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:05.274873 15804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:05.275097 15804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:05.275122 15804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:05.276294 15804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:05.276340 15804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:05.276368 15804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:13:05.278052 15804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:42787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:05.278344 15804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:05.278616 15804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:05.284395 15812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.284344 15818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.284441 15814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:05.284487 15804 server_base.cc:1034] running on GCE node
I20260812 01:13:05.284854 15804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:05.285229 15804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:05.286402 15804 hybrid_clock.cc:648] HybridClock initialized: now 1786497185286377 us; error 37 us; skew 500 ppm
Aug 12 01:13:05 dist-test-slave-tfpf krb5kdc[15386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:05.289433 15804 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:13:05.289973 15804 webserver.cc:460] Webserver started at http://127.17.130.2:37329/ using document root <none> and password file <none>
I20260812 01:13:05.290256 15804 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:05.290329 15804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:05.290480 15804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:05.291635 15804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "02781e993043465dbafcb29e948f60ed"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "d240265238f23e7f1ca9e7fd6c530949"
server_key_iv: "460d255921c843dc986264ca6237746e"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.292018 15804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "02781e993043465dbafcb29e948f60ed"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "d240265238f23e7f1ca9e7fd6c530949"
server_key_iv: "460d255921c843dc986264ca6237746e"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.293643 15804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:05.294550 15830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.294788 15804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:05.294893 15804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "02781e993043465dbafcb29e948f60ed"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "d240265238f23e7f1ca9e7fd6c530949"
server_key_iv: "460d255921c843dc986264ca6237746e"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.294981 15804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:05.320173 15804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:05.322690 15804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:05.322861 15804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:05.323128 15804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:05.323457 15804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:05.323518 15804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.323578 15804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:05.323606 15804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.329912 15804 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:43197
I20260812 01:13:05.329965 16026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:43197 every 8 connection(s)
I20260812 01:13:05.330302 15804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:13:05.332120 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 15804
I20260812 01:13:05.332249 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:13:05.332512 17928 external_mini_cluster.cc:1428] Setting key f86a0c7812d814553683cdd746792363
Aug 12 01:13:05 dist-test-slave-tfpf krb5kdc[15386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:05.342803 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.331028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:42703 (local address 127.17.130.62:42787)
0812 01:13:05.331106 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.331108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.331291 (+ 183us) server_negotiation.cc:407] Connection header received
0812 01:13:05.331833 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.331835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.331846 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.331932 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:05.333340 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.333860 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.335506 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.335844 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.335926 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.338863 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:05.338865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:05.338866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:05.338875 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:05.340951 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.341485 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.341487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.341488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.341563 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.341953 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.341954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.341954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.342164 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:05.342249 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.342575 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.342675 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 01:13:05.342831 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.330994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:42787 (local address 127.17.130.2:42703)
0812 01:13:05.331174 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.331216 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.331717 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.332060 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.332066 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.332921 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:05.333246 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.333254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.333958 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.333959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.335295 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.335299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.335955 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.335956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.336087 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.336727 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:05.336737 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:05.338743 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:05.341113 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:05.341123 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:05.341126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:05.341402 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:05.341663 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:05.341664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:05.341664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:05.341796 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:05.342250 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:05.342257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:05.342395 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.342503 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.342648 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260812 01:13:05.343495 16028 heartbeater.cc:347] Connected to a master server at 127.17.130.62:42787
I20260812 01:13:05.343636 16028 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
I20260812 01:13:05.343839 16028 heartbeater.cc:511] Master 127.17.130.62:42787 requested a full tablet report, sending...
I20260812 01:13:05.344275 15457 ts_manager.cc:194] Registered new tserver with Master: 02781e993043465dbafcb29e948f60ed (127.17.130.2:43197)
I20260812 01:13:05.344697 15457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:42703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:13:05.364209 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:42787
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:05.456640 16035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:05.456881 16035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:05.456907 16035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:05.458271 16035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:05.458328 16035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:05.458357 16035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:13:05.460326 16035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:42787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:05.460786 16035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:05.461241 16035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:05.468220 16048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.468245 16043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.468258 16044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:05.468513 16035 server_base.cc:1034] running on GCE node
I20260812 01:13:05.468765 16035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:05.469110 16035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:05.470316 16035 hybrid_clock.cc:648] HybridClock initialized: now 1786497185470291 us; error 47 us; skew 500 ppm
Aug 12 01:13:05 dist-test-slave-tfpf krb5kdc[15386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:05.473278 16035 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:13:05.473771 16035 webserver.cc:460] Webserver started at http://127.17.130.3:38977/ using document root <none> and password file <none>
I20260812 01:13:05.473973 16035 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:05.474014 16035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:05.474115 16035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:05.475179 16035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "f7a5d7b191e748a0979548bbf27af0d7"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "f541ee35f4c4545745e61c07d8aa74ea"
server_key_iv: "7e48ac97eb0c4b14d4d54043e095be42"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.475502 16035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "f7a5d7b191e748a0979548bbf27af0d7"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "f541ee35f4c4545745e61c07d8aa74ea"
server_key_iv: "7e48ac97eb0c4b14d4d54043e095be42"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.477942 16035 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260812 01:13:05.478775 16059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.479048 16035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:05.479139 16035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "f7a5d7b191e748a0979548bbf27af0d7"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "f541ee35f4c4545745e61c07d8aa74ea"
server_key_iv: "7e48ac97eb0c4b14d4d54043e095be42"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.479202 16035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:05.519412 16035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:05.521652 16035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:05.521781 16035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:05.522233 16035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:05.522578 16035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:05.522619 16035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.522717 16035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:05.522785 16035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.530819 16035 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:45125
I20260812 01:13:05.531028 16261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:45125 every 8 connection(s)
I20260812 01:13:05.531334 16035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:13:05.531502 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 16035
I20260812 01:13:05.531618 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:13:05.531828 17928 external_mini_cluster.cc:1428] Setting key df6bc41fdeee7e7d6fcc362df2805ec0
Aug 12 01:13:05 dist-test-slave-tfpf krb5kdc[15386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497185, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:05.542513 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.532267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:39145 (local address 127.17.130.62:42787)
0812 01:13:05.532340 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.532343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.532533 (+ 190us) server_negotiation.cc:407] Connection header received
0812 01:13:05.533111 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.533112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.533126 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.533191 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:05.534217 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.534751 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.536073 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.536278 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.536337 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.539223 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:05.539225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:05.539225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:05.539235 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:05.540772 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.541386 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.541388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.541388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.541452 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.541786 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.541787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.541787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.541932 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:05.542001 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.542359 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.542440 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:13:05.542691 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.532210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:42787 (local address 127.17.130.3:39145)
0812 01:13:05.532412 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.532455 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.532945 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.533268 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.533271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.533881 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:05.534101 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.534106 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.534888 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.534889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.535897 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.535900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.536373 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.536374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.536493 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.537305 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:05.537314 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:05.539099 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:05.540882 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:05.540891 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:05.540894 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:05.541289 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:05.541574 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:05.541575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:05.541575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:05.541702 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:05.542023 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:05.542030 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:05.542190 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.542296 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.542446 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260812 01:13:05.543318 16263 heartbeater.cc:347] Connected to a master server at 127.17.130.62:42787
I20260812 01:13:05.543442 16263 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:05.543707 16263 heartbeater.cc:511] Master 127.17.130.62:42787 requested a full tablet report, sending...
I20260812 01:13:05.544202 15457 ts_manager.cc:194] Registered new tserver with Master: f7a5d7b191e748a0979548bbf27af0d7 (127.17.130.3:45125)
I20260812 01:13:05.544592 15457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:39145
I20260812 01:13:05.555361 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:13:05.565644 15535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.557203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.17.130.62:42787)
0812 01:13:05.557272 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.557274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.557286 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:13:05.557316 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.557317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.557327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.557389 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:05.557995 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.558514 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.559239 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.559439 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.559513 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.561417 (+ 1904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:05.561418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:05.561419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:05.561427 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:05.563837 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.564634 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.564635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.564636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.564704 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:05.564953 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:05.564954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:05.564954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:05.565185 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:05.565312 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.565454 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.565529 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:13:05.567924 15457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:13:05.569960 15457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:13:05.578773 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.572995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:45125 (local address 127.0.0.1:36704)
0812 01:13:05.573299 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.573346 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.573460 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.573834 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.573838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.573841 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:05.573934 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.573942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.576037 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.576040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.577563 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.577567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.578468 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.578469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.578574 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.578578 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.578612 (+ 34us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.578675 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":177,"thread_start_us":53,"threads_started":1}
I20260812 01:13:05.578886 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.573426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46512 (local address 127.17.130.2:43197)
0812 01:13:05.573637 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.573641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.573654 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:13:05.573919 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.573923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.573943 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.574030 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:05.574274 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.575860 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.577690 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.578494 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.578583 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.578638 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.578746 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.578786 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260812 01:13:05.579084 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.573015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:43197 (local address 127.0.0.1:46512)
0812 01:13:05.573601 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.573611 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.573656 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.574118 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.574119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.574122 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:05.574181 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.574184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.576038 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.576040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.577579 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.577583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.578590 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.578591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.578663 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.578667 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.578728 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.578777 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":296,"thread_start_us":218,"threads_started":1}
I20260812 01:13:05.579282 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.573291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58418 (local address 127.17.130.1:38529)
0812 01:13:05.573762 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.573766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.573803 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:13:05.573989 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.573991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.574009 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.574281 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:05.574482 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.575244 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.577462 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.578247 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.578337 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.578402 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.578840 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.579093 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":162,"threads_started":1}
I20260812 01:13:05.578773 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.573015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:38529 (local address 127.0.0.1:58418)
0812 01:13:05.573751 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.573760 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.573810 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.574360 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.574361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.574363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:05.574394 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.574396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.575470 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.575471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.577346 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.577350 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.578356 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.578357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.578463 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.578482 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.578564 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.578637 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":2,"spinlock_wait_cycles":2304,"thread_start_us":218,"threads_started":1}
I20260812 01:13:05.579706 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.573437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.17.130.3:45125)
0812 01:13:05.573629 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.573635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.573645 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:05.573689 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.573692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.573711 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.574043 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:05.574115 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.575743 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.577691 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.578276 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.579279 (+ 1003us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.579362 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.579555 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.579620 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260812 01:13:05.579988 15921 tablet_service.cc:1467] Processing CreateTablet for tablet 07acdc3d545e422c8ee5efa8e53a429d (DEFAULT_TABLE table=test-table [id=90fa02f974d8475e9c1bd39e0f09f370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:05.580109 15677 tablet_service.cc:1467] Processing CreateTablet for tablet 07acdc3d545e422c8ee5efa8e53a429d (DEFAULT_TABLE table=test-table [id=90fa02f974d8475e9c1bd39e0f09f370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:05.580436 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07acdc3d545e422c8ee5efa8e53a429d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:05.580436 15921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07acdc3d545e422c8ee5efa8e53a429d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:05.581732 16136 tablet_service.cc:1467] Processing CreateTablet for tablet 07acdc3d545e422c8ee5efa8e53a429d (DEFAULT_TABLE table=test-table [id=90fa02f974d8475e9c1bd39e0f09f370]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:05.582137 16136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07acdc3d545e422c8ee5efa8e53a429d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:05.584316 16288 tablet_bootstrap.cc:492] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: Bootstrap starting.
I20260812 01:13:05.584980 16288 tablet_bootstrap.cc:654] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:05.585605 16290 tablet_bootstrap.cc:492] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Bootstrap starting.
I20260812 01:13:05.585647 16288 log.cc:824] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:05.586515 16290 tablet_bootstrap.cc:654] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:05.586684 16288 tablet_bootstrap.cc:492] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: No bootstrap required, opened a new log
I20260812 01:13:05.586771 16288 ts_tablet_manager.cc:1397] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:13:05.587073 16290 log.cc:824] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:05.587774 16290 tablet_bootstrap.cc:492] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: No bootstrap required, opened a new log
I20260812 01:13:05.587859 16290 ts_tablet_manager.cc:1397] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:05.588353 16293 tablet_bootstrap.cc:492] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Bootstrap starting.
I20260812 01:13:05.588894 16288 raft_consensus.cc:348] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.589107 16288 raft_consensus.cc:374] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:05.589160 16288 raft_consensus.cc:729] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c745bb4c135947708d96adafffc6637d, State: Initialized, Role: FOLLOWER
I20260812 01:13:05.589258 16293 tablet_bootstrap.cc:654] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:05.589283 16288 consensus_queue.cc:260] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.589608 15795 heartbeater.cc:503] Master 127.17.130.62:42787 was elected leader, sending a full tablet report...
I20260812 01:13:05.589888 16293 log.cc:824] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:05.589618 16288 ts_tablet_manager.cc:1428] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:05.590327 16290 raft_consensus.cc:348] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.590552 16290 raft_consensus.cc:374] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:05.590598 16290 raft_consensus.cc:729] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7a5d7b191e748a0979548bbf27af0d7, State: Initialized, Role: FOLLOWER
I20260812 01:13:05.590778 16293 tablet_bootstrap.cc:492] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: No bootstrap required, opened a new log
I20260812 01:13:05.590744 16290 consensus_queue.cc:260] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.590857 16293 ts_tablet_manager.cc:1397] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 01:13:05.591071 16290 ts_tablet_manager.cc:1428] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:05.592478 16293 raft_consensus.cc:348] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.592628 16293 raft_consensus.cc:374] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:05.592653 16293 raft_consensus.cc:729] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02781e993043465dbafcb29e948f60ed, State: Initialized, Role: FOLLOWER
I20260812 01:13:05.592753 16293 consensus_queue.cc:260] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.593106 16293 ts_tablet_manager.cc:1428] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:05.593150 16263 heartbeater.cc:503] Master 127.17.130.62:42787 was elected leader, sending a full tablet report...
I20260812 01:13:05.593194 16028 heartbeater.cc:503] Master 127.17.130.62:42787 was elected leader, sending a full tablet report...
W20260812 01:13:05.646013 15796 tablet.cc:2369] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:05.646852 16306 raft_consensus.cc:482] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:13:05.646963 16306 raft_consensus.cc:504] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.647364 16306 leader_election.cc:288] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7a5d7b191e748a0979548bbf27af0d7 (127.17.130.3:45125), c745bb4c135947708d96adafffc6637d (127.17.130.1:38529)
I20260812 01:13:05.651230 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.647655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:35617 (local address 127.17.130.1:38529)
0812 01:13:05.647715 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.647717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.647979 (+ 262us) server_negotiation.cc:407] Connection header received
0812 01:13:05.648011 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.648012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.648022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.648099 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:05.648286 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.648777 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.650203 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.650773 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.650863 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.650872 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.651103 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.651157 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:13:05.651230 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.647645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:60639 (local address 127.17.130.3:45125)
0812 01:13:05.647718 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.647721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.647731 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:05.647820 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.647827 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.647838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.647919 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:05.648147 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.648706 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.650203 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.650796 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.650863 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.650872 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.651109 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.651151 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 01:13:05.651249 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.647573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:45125 (local address 127.17.130.2:60639)
0812 01:13:05.647657 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.647670 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.647724 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.647981 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.647982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.647985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:05.648073 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.648075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.648790 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.648791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.650083 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.650086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.650895 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.650896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.651011 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.651013 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.651102 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.651139 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":31}
I20260812 01:13:05.651700 16170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07acdc3d545e422c8ee5efa8e53a429d" candidate_uuid: "02781e993043465dbafcb29e948f60ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a5d7b191e748a0979548bbf27af0d7" is_pre_election: true
I20260812 01:13:05.651257 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.647577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:38529 (local address 127.17.130.2:35617)
0812 01:13:05.647857 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.647868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:05.647928 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:05.648102 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:05.648102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:05.648104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:05.648214 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.648215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.648846 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.648847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.650083 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:05.650086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.650895 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.650896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:05.651006 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.651012 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 01:13:05.651081 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:13:05.651134 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":111,"thread_start_us":109,"threads_started":1}
I20260812 01:13:05.651889 16170 raft_consensus.cc:2454] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02781e993043465dbafcb29e948f60ed in term 0.
I20260812 01:13:05.652155 15837 leader_election.cc:302] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02781e993043465dbafcb29e948f60ed, f7a5d7b191e748a0979548bbf27af0d7; no voters:
I20260812 01:13:05.652145 15716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07acdc3d545e422c8ee5efa8e53a429d" candidate_uuid: "02781e993043465dbafcb29e948f60ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c745bb4c135947708d96adafffc6637d" is_pre_election: true
I20260812 01:13:05.652307 15716 raft_consensus.cc:2454] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02781e993043465dbafcb29e948f60ed in term 0.
I20260812 01:13:05.652318 16306 raft_consensus.cc:2781] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:13:05.652392 16306 raft_consensus.cc:482] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:13:05.652426 16306 raft_consensus.cc:3037] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:05.653244 16306 raft_consensus.cc:504] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.653403 16306 leader_election.cc:288] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [CANDIDATE]: Term 1 election: Requested vote from peers f7a5d7b191e748a0979548bbf27af0d7 (127.17.130.3:45125), c745bb4c135947708d96adafffc6637d (127.17.130.1:38529)
I20260812 01:13:05.653594 15716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07acdc3d545e422c8ee5efa8e53a429d" candidate_uuid: "02781e993043465dbafcb29e948f60ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c745bb4c135947708d96adafffc6637d"
I20260812 01:13:05.653594 16170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07acdc3d545e422c8ee5efa8e53a429d" candidate_uuid: "02781e993043465dbafcb29e948f60ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7a5d7b191e748a0979548bbf27af0d7"
I20260812 01:13:05.653690 16170 raft_consensus.cc:3037] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:05.653690 15716 raft_consensus.cc:3037] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:05.654636 16170 raft_consensus.cc:2454] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02781e993043465dbafcb29e948f60ed in term 1.
I20260812 01:13:05.654636 15716 raft_consensus.cc:2454] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02781e993043465dbafcb29e948f60ed in term 1.
I20260812 01:13:05.654841 15837 leader_election.cc:302] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02781e993043465dbafcb29e948f60ed, f7a5d7b191e748a0979548bbf27af0d7; no voters:
I20260812 01:13:05.654968 16306 raft_consensus.cc:2781] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:05.655256 16306 raft_consensus.cc:686] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 1 LEADER]: Becoming Leader. State: Replica: 02781e993043465dbafcb29e948f60ed, State: Running, Role: LEADER
I20260812 01:13:05.655412 16306 consensus_queue.cc:237] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } }
I20260812 01:13:05.656418 15457 catalog_manager.cc:5495] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 02781e993043465dbafcb29e948f60ed (127.17.130.2). New cstate: current_term: 1 leader_uuid: "02781e993043465dbafcb29e948f60ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02781e993043465dbafcb29e948f60ed" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 43197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 } health_report { overall_health: UNKNOWN } } }
I20260812 01:13:05.673168 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.670445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.17.130.2:43197)
0812 01:13:05.670549 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.670551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.670561 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:05.670600 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.670601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.670622 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.670694 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:05.670807 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.671307 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.672085 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.672340 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.672409 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.672608 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:05.672701 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:05.672780 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.672974 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.672992 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 01:13:05.676061 16170 raft_consensus.cc:1261] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 1 FOLLOWER]: Refusing update from remote peer 02781e993043465dbafcb29e948f60ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:05.676047 15716 raft_consensus.cc:1261] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 1 FOLLOWER]: Refusing update from remote peer 02781e993043465dbafcb29e948f60ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:05.676538 16310 consensus_queue.cc:1035] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "c745bb4c135947708d96adafffc6637d" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 38529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:05.676719 16306 consensus_queue.cc:1035] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7a5d7b191e748a0979548bbf27af0d7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 45125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:05.679600 16319 mvcc.cc:205] Tried to move back new op lower bound from 7317492472524435456 to 7317492472445186048. Current Snapshot: MvccSnapshot[applied={T|T < 7317492472524435456}]
I20260812 01:13:05.683351 15457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:05.683466 15457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55338:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:05.684485 15457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80fe137164614924986e3c2ccbd9190c: Sending DeleteTablet for 3 replicas of tablet 07acdc3d545e422c8ee5efa8e53a429d
I20260812 01:13:05.684931 16136 tablet_service.cc:1514] Processing DeleteTablet for tablet 07acdc3d545e422c8ee5efa8e53a429d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:05 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:36704
I20260812 01:13:05.684942 15677 tablet_service.cc:1514] Processing DeleteTablet for tablet 07acdc3d545e422c8ee5efa8e53a429d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:05 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:58418
I20260812 01:13:05.685227 15921 tablet_service.cc:1514] Processing DeleteTablet for tablet 07acdc3d545e422c8ee5efa8e53a429d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:05 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:46512
I20260812 01:13:05.685508 16338 tablet_replica.cc:330] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: stopping tablet replica
I20260812 01:13:05.685591 16336 tablet_replica.cc:330] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: stopping tablet replica
I20260812 01:13:05.685675 16338 raft_consensus.cc:2229] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:13:05.685788 16336 raft_consensus.cc:2229] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:05.685839 16338 raft_consensus.cc:2258] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:05.685937 16336 raft_consensus.cc:2258] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:05.686152 16337 tablet_replica.cc:330] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d: stopping tablet replica
I20260812 01:13:05.686260 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 15569
I20260812 01:13:05.686314 16337 raft_consensus.cc:2229] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:05.686434 16337 raft_consensus.cc:2258] T 07acdc3d545e422c8ee5efa8e53a429d P c745bb4c135947708d96adafffc6637d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:05.686770 16336 ts_tablet_manager.cc:1905] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:05.688777 16336 ts_tablet_manager.cc:1918] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:05.689021 16336 log.cc:1196] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/07acdc3d545e422c8ee5efa8e53a429d
I20260812 01:13:05.689570 16336 ts_tablet_manager.cc:1939] T 07acdc3d545e422c8ee5efa8e53a429d P f7a5d7b191e748a0979548bbf27af0d7: Deleting consensus metadata
I20260812 01:13:05.690251 15434 catalog_manager.cc:4841] TS f7a5d7b191e748a0979548bbf27af0d7 (127.17.130.3:45125): tablet 07acdc3d545e422c8ee5efa8e53a429d (table test-table [id=90fa02f974d8475e9c1bd39e0f09f370]) successfully deleted
I20260812 01:13:05.692728 16338 ts_tablet_manager.cc:1905] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:05.694794 16338 ts_tablet_manager.cc:1918] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:05.695015 16338 log.cc:1196] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/07acdc3d545e422c8ee5efa8e53a429d
I20260812 01:13:05.695399 16338 ts_tablet_manager.cc:1939] T 07acdc3d545e422c8ee5efa8e53a429d P 02781e993043465dbafcb29e948f60ed: Deleting consensus metadata
I20260812 01:13:05.696153 15433 catalog_manager.cc:4841] TS 02781e993043465dbafcb29e948f60ed (127.17.130.2:43197): tablet 07acdc3d545e422c8ee5efa8e53a429d (table test-table [id=90fa02f974d8475e9c1bd39e0f09f370]) successfully deleted
W20260812 01:13:05.705817 15435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:38529 (error 108)
I20260812 01:13:05.706137 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 15804
I20260812 01:13:05.706899 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.706379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:38529 (local address 127.0.0.1:58426)
0812 01:13:05.706801 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:05.706859 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:38529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389}
W20260812 01:13:05.709867 15435 catalog_manager.cc:4584] TS c745bb4c135947708d96adafffc6637d (127.17.130.1:38529): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07acdc3d545e422c8ee5efa8e53a429d: Network error: Client connection negotiation failed: client connection to 127.17.130.1:38529: connect: Connection refused (error 111)
I20260812 01:13:05.721190 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 16035
I20260812 01:13:05.747275 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 15410
2026-08-12T01:13:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2550 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T01:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:05Z Disabled control of system clock
I20260812 01:13:05.796875 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:35311
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:35311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:05.890798 16349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:05.891052 16349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:05.891076 16349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:05.892400 16349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:05.892452 16349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:05.892464 16349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:05.892477 16349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:05.892490 16349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:05.894029 16349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:35311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:35311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:05.894322 16349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:05.894681 16349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:05.900126 16359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.900051 16356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:05.900058 16361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:05.900769 16349 server_base.cc:1034] running on GCE node
I20260812 01:13:05.901288 16349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:05.901662 16349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:05.902848 16349 hybrid_clock.cc:648] HybridClock initialized: now 1786497185902770 us; error 109 us; skew 500 ppm
I20260812 01:13:05.904449 16349 webserver.cc:460] Webserver started at http://127.17.130.62:40685/ using document root <none> and password file <none>
I20260812 01:13:05.904768 16349 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:05.904843 16349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:05.904999 16349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:05.906231 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "b9a490db44f44cd59f909bb45e2baae8"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "e5acebddc1a8392fa9ba3659ec639993"
server_key_iv: "83f7c310e5164f8855c3f7bdc8eb374d"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.906649 16349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "b9a490db44f44cd59f909bb45e2baae8"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "e5acebddc1a8392fa9ba3659ec639993"
server_key_iv: "83f7c310e5164f8855c3f7bdc8eb374d"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.908202 16349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 01:13:05.909242 16371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:05.909436 16349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:05.909561 16349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "b9a490db44f44cd59f909bb45e2baae8"
format_stamp: "Formatted at 2026-08-12 01:13:05 on dist-test-slave-tfpf"
server_key: "e5acebddc1a8392fa9ba3659ec639993"
server_key_iv: "83f7c310e5164f8855c3f7bdc8eb374d"
server_key_version: "encryptionkey@0"
I20260812 01:13:05.909647 16349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:05.920419 16349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:05.920809 16349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:05.920972 16349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:05.925823 16349 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:35311
I20260812 01:13:05.925858 16459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:35311 every 8 connection(s)
I20260812 01:13:05.926254 16349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:05.927345 16460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:05.930132 16460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8: Bootstrap starting.
I20260812 01:13:05.930847 16460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:05.931192 16460 log.cc:824] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:05.932058 16460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8: No bootstrap required, opened a new log
I20260812 01:13:05.933323 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 16349
I20260812 01:13:05.933462 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:05.933722 17928 external_mini_cluster.cc:1428] Setting key cf86c1f7eb82130583901c73c649b3b9
I20260812 01:13:05.934204 16460 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a490db44f44cd59f909bb45e2baae8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 35311 } }
I20260812 01:13:05.934418 16460 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:05.934460 16460 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9a490db44f44cd59f909bb45e2baae8, State: Initialized, Role: FOLLOWER
I20260812 01:13:05.934636 16460 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a490db44f44cd59f909bb45e2baae8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 35311 } }
I20260812 01:13:05.934758 16460 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:05.934801 16460 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:05.934854 16460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:05.935602 16460 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a490db44f44cd59f909bb45e2baae8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 35311 } }
I20260812 01:13:05.935760 16460 leader_election.cc:302] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9a490db44f44cd59f909bb45e2baae8; no voters:
I20260812 01:13:05.935981 16460 leader_election.cc:288] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:05.936018 16465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:05.936167 16465 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [term 1 LEADER]: Becoming Leader. State: Replica: b9a490db44f44cd59f909bb45e2baae8, State: Running, Role: LEADER
I20260812 01:13:05.936311 16460 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:05.936277 16465 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a490db44f44cd59f909bb45e2baae8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 35311 } }
I20260812 01:13:05.936728 16465 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9a490db44f44cd59f909bb45e2baae8. Latest consensus state: current_term: 1 leader_uuid: "b9a490db44f44cd59f909bb45e2baae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a490db44f44cd59f909bb45e2baae8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 35311 } } }
I20260812 01:13:05.936838 16465 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:05.936740 16467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9a490db44f44cd59f909bb45e2baae8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9a490db44f44cd59f909bb45e2baae8" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 35311 } } }
I20260812 01:13:05.937032 16467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:05.937171 16473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:05.937808 16473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:05.939924 16473 catalog_manager.cc:1347] Generated new cluster ID: b90dd53c98344ba68ff7743998432112
I20260812 01:13:05.940089 16473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:05.941231 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:05.934565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39708 (local address 127.17.130.62:35311)
0812 01:13:05.934948 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:05.934959 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:05.935004 (+ 45us) server_negotiation.cc:407] Connection header received
0812 01:13:05.935134 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:05.935140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:05.935204 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:05.935397 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:05.935684 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.937580 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.938837 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:05.939280 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:05.939822 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:05.940240 (+ 418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:05.940244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:05.940244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:05.940507 (+ 263us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:05.940540 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:05.940553 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:05.940662 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:05.940921 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 01:13:05.941010 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":143,"threads_started":1}
I20260812 01:13:05.964318 16473 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:05.964989 16473 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:05.969092 16473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8: Generated new TSK 0
I20260812 01:13:05.969347 16473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:05.974530 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:35311
--builtin_ntp_servers=127.17.130.20:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:06.072476 16491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:06.072772 16491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:06.072816 16491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:06.074240 16491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:06.074331 16491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:06.074383 16491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:13:06.075994 16491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:35311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:06.076323 16491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:06.076704 16491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:06.082590 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:06.082610 16507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:06.082857 16512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:06.083045 16491 server_base.cc:1034] running on GCE node
I20260812 01:13:06.083268 16491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:06.083531 16491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:06.084712 16491 hybrid_clock.cc:648] HybridClock initialized: now 1786497186084683 us; error 47 us; skew 500 ppm
I20260812 01:13:06.086318 16491 webserver.cc:460] Webserver started at http://127.17.130.1:38797/ using document root <none> and password file <none>
I20260812 01:13:06.086625 16491 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:06.086702 16491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:06.086849 16491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:06.088049 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "79e1e90a9abd425fa8da22077f76ef8a"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "57849be50058512a27a3acc955c1a92e"
server_key_iv: "e0842d95719efb38559d4e2d671ca982"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.088454 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "79e1e90a9abd425fa8da22077f76ef8a"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "57849be50058512a27a3acc955c1a92e"
server_key_iv: "e0842d95719efb38559d4e2d671ca982"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.090445 16491 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 01:13:06.091377 16520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.091612 16491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:06.091743 16491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "79e1e90a9abd425fa8da22077f76ef8a"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "57849be50058512a27a3acc955c1a92e"
server_key_iv: "e0842d95719efb38559d4e2d671ca982"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.091836 16491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:06.103996 16491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:06.104400 16491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:06.104573 16491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:06.104846 16491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:06.105304 16491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:06.105361 16491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.105408 16491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:06.105443 16491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.112509 16491 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:45579
I20260812 01:13:06.112653 16706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:45579 every 8 connection(s)
I20260812 01:13:06.113301 16491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:13:06.119975 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.114255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:53759 (local address 127.17.130.62:35311)
0812 01:13:06.114354 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.114358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.114619 (+ 261us) server_negotiation.cc:407] Connection header received
0812 01:13:06.115209 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.115210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.115227 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.115303 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:06.116152 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.116699 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.117977 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.118261 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.118462 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.119601 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:06.119602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:06.119602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:06.119613 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:06.119632 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:06.119637 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:06.119699 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.119850 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.119895 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 01:13:06.120206 16710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.114222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:35311 (local address 127.17.130.1:53759)
0812 01:13:06.114457 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.114499 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.115091 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.115424 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.115428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.115438 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:06.116031 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.116036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.116803 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.116804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.117773 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.117776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.118490 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.118491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.118640 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.119399 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:06.119400 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:06.119435 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:06.119465 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:06.119742 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:06.119749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:06.119752 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.119827 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.119938 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":118,"threads_started":1}
I20260812 01:13:06.121114 16707 heartbeater.cc:347] Connected to a master server at 127.17.130.62:35311
I20260812 01:13:06.121254 16707 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:06.121500 16707 heartbeater.cc:511] Master 127.17.130.62:35311 requested a full tablet report, sending...
I20260812 01:13:06.122069 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 16491
I20260812 01:13:06.122184 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:13:06.122279 16400 ts_manager.cc:194] Registered new tserver with Master: 79e1e90a9abd425fa8da22077f76ef8a (127.17.130.1:45579)
I20260812 01:13:06.122431 17928 external_mini_cluster.cc:1428] Setting key 7daeb1cf2a727b000d8986e37feb8304
I20260812 01:13:06.123217 16400 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.1:53759
I20260812 01:13:06.124305 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:35311
--builtin_ntp_servers=127.17.130.20:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:06.220832 16712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:06.221194 16712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:06.221227 16712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:06.222522 16712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:06.222586 16712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:06.222620 16712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:13:06.224218 16712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:35311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:06.224573 16712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:06.224963 16712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:06.230762 16725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:06.230849 16722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:06.230783 16719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:06.230937 16712 server_base.cc:1034] running on GCE node
I20260812 01:13:06.231237 16712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:06.231511 16712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:06.232718 16712 hybrid_clock.cc:648] HybridClock initialized: now 1786497186232715 us; error 55 us; skew 500 ppm
I20260812 01:13:06.234412 16712 webserver.cc:460] Webserver started at http://127.17.130.2:34095/ using document root <none> and password file <none>
I20260812 01:13:06.234706 16712 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:06.234759 16712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:06.234936 16712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:06.236089 16712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "a05fc67646c74811985f65f57dc8a6b6"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "a2d108325015096637302b25633cbd38"
server_key_iv: "83ffe590f23280c0978ebf0fd24377d4"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.236502 16712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "a05fc67646c74811985f65f57dc8a6b6"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "a2d108325015096637302b25633cbd38"
server_key_iv: "83ffe590f23280c0978ebf0fd24377d4"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.238438 16712 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 01:13:06.239452 16739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.239893 16712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:06.240130 16712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "a05fc67646c74811985f65f57dc8a6b6"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "a2d108325015096637302b25633cbd38"
server_key_iv: "83ffe590f23280c0978ebf0fd24377d4"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.240270 16712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:06.264293 16712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:06.264700 16712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:06.264887 16712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:06.265214 16712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:06.265640 16712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:06.265699 16712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.265748 16712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:06.265798 16712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.272238 16712 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:40931
I20260812 01:13:06.272393 16928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:40931 every 8 connection(s)
I20260812 01:13:06.272715 16712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:13:06.279028 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.273714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:47815 (local address 127.17.130.62:35311)
0812 01:13:06.273815 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.273817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.274033 (+ 216us) server_negotiation.cc:407] Connection header received
0812 01:13:06.274794 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.274795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.274811 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.274885 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:06.275654 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.276189 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.277330 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.277535 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.277625 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.278654 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:06.278655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:06.278656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:06.278662 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:06.278679 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:06.278685 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:06.278754 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.278870 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.278907 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 01:13:06.279235 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.273568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:35311 (local address 127.17.130.2:47815)
0812 01:13:06.273854 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.273907 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.274644 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.274962 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.274966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.274976 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:06.275544 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.275549 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.276301 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.276303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.277219 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.277223 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.277641 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.277643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.277836 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.278481 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:06.278481 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:06.278513 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:06.278540 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:06.278769 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:06.278777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:06.278780 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.278871 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.278987 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
I20260812 01:13:06.279800 16929 heartbeater.cc:347] Connected to a master server at 127.17.130.62:35311
I20260812 01:13:06.279908 16929 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:06.280084 16929 heartbeater.cc:511] Master 127.17.130.62:35311 requested a full tablet report, sending...
I20260812 01:13:06.280481 16400 ts_manager.cc:194] Registered new tserver with Master: a05fc67646c74811985f65f57dc8a6b6 (127.17.130.2:40931)
I20260812 01:13:06.280879 16400 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.2:47815
I20260812 01:13:06.282730 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 16712
I20260812 01:13:06.282820 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:13:06.283108 17928 external_mini_cluster.cc:1428] Setting key 88fb22187a3f234c1d1a010f49169712
I20260812 01:13:06.284335 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:35311
--builtin_ntp_servers=127.17.130.20:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:06.375093 16936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:06.375330 16936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:06.375355 16936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:06.376500 16936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:06.376540 16936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:06.376566 16936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:13:06.378038 16936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:35311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:06.378281 16936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:06.378568 16936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:06.384074 16947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:06.384060 16950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:06.384065 16948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:06.384362 16936 server_base.cc:1034] running on GCE node
I20260812 01:13:06.384616 16936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:06.384873 16936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:06.386044 16936 hybrid_clock.cc:648] HybridClock initialized: now 1786497186386030 us; error 31 us; skew 500 ppm
I20260812 01:13:06.387450 16936 webserver.cc:460] Webserver started at http://127.17.130.3:40737/ using document root <none> and password file <none>
I20260812 01:13:06.387727 16936 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:06.387800 16936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:06.387952 16936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:06.389173 16936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "ada99532b0c34126b29a04ee0c8709f5"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "159912dfde2b8ba9567db3df7635490f"
server_key_iv: "a226799ce0938500b51051d13b96f45a"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.389547 16936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "ada99532b0c34126b29a04ee0c8709f5"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "159912dfde2b8ba9567db3df7635490f"
server_key_iv: "a226799ce0938500b51051d13b96f45a"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.391469 16936 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:06.392395 16958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.392652 16936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:06.392765 16936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "ada99532b0c34126b29a04ee0c8709f5"
format_stamp: "Formatted at 2026-08-12 01:13:06 on dist-test-slave-tfpf"
server_key: "159912dfde2b8ba9567db3df7635490f"
server_key_iv: "a226799ce0938500b51051d13b96f45a"
server_key_version: "encryptionkey@0"
I20260812 01:13:06.392858 16936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:06.411123 16936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:06.411507 16936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:06.411686 16936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:06.411952 16936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:06.412351 16936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:06.412403 16936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.412446 16936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:06.412500 16936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:06.419204 16936 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:37035
I20260812 01:13:06.419299 17158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:37035 every 8 connection(s)
I20260812 01:13:06.419687 16936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:13:06.421269 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 16936
I20260812 01:13:06.421363 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:13:06.421557 17928 external_mini_cluster.cc:1428] Setting key 3fb338f5f401a1837c5799f55c1f6325
I20260812 01:13:06.426075 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.420537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:45215 (local address 127.17.130.62:35311)
0812 01:13:06.420586 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.420588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.420939 (+ 351us) server_negotiation.cc:407] Connection header received
0812 01:13:06.421830 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.421831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.421845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.421913 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:06.422906 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.423427 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.424445 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.424650 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.424719 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.425721 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:06.425722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:06.425723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:06.425728 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:06.425744 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:06.425748 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:06.425803 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.425962 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.426010 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:06.426254 17167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.420468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:35311 (local address 127.17.130.3:45215)
0812 01:13:06.420685 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.420729 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.421719 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.422028 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.422033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.422044 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:06.422802 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.422808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.423540 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.423541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.424338 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.424340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.424791 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.424792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.424935 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.425581 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:06.425581 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:06.425612 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:06.425638 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:06.425844 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:06.425851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:06.425855 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.425945 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.426036 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260812 01:13:06.426891 17161 heartbeater.cc:347] Connected to a master server at 127.17.130.62:35311
I20260812 01:13:06.427027 17161 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:06.427260 17161 heartbeater.cc:511] Master 127.17.130.62:35311 requested a full tablet report, sending...
I20260812 01:13:06.427736 16400 ts_manager.cc:194] Registered new tserver with Master: ada99532b0c34126b29a04ee0c8709f5 (127.17.130.3:37035)
I20260812 01:13:06.428186 16400 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.3:45215
I20260812 01:13:06.433701 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:13:06.439361 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.435317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38636 (local address 127.17.130.62:35311)
0812 01:13:06.435386 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.435388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.435463 (+ 75us) server_negotiation.cc:407] Connection header received
0812 01:13:06.435515 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.435516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.435530 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.435577 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:06.435788 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.436463 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.437624 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.437997 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.438164 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.439019 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:06.439021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:06.439021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:06.439027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:06.439045 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:06.439050 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:06.439131 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.439213 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.439292 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:13:06.441416 16400 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:13:06.443089 16400 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:13:06.450124 17174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.445604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:45579 (local address 127.0.0.1:37600)
0812 01:13:06.445870 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.445883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.445943 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.446409 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.446410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.446413 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:06.446492 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.446493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.447833 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.447834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.448821 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.448823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.449762 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.449763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.449855 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.449872 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.449967 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.450030 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":30,"thread_start_us":67,"threads_started":1}
I20260812 01:13:06.450966 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.445604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37035 (local address 127.0.0.1:49468)
0812 01:13:06.445794 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.445837 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.445909 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.446420 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.446421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.446422 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:06.446491 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.446493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.448588 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.448589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.449882 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.449886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.450711 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.450712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.450779 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.450784 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.450848 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.450891 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":3,"thread_start_us":70,"threads_started":1}
I20260812 01:13:06.450995 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.445999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.17.130.1:45579)
0812 01:13:06.446199 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.446203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.446212 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:06.446252 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.446254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.446270 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.446344 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:06.446563 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.447684 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.448909 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.449623 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.449751 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.449856 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.450794 (+ 938us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.450905 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260812 01:13:06.451381 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.445801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.17.130.3:37035)
0812 01:13:06.446211 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.446215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.446223 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:13:06.446276 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.446278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.446295 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.446418 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:06.447799 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.448480 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.449968 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.450622 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.450691 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.450750 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.450944 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.450990 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":98,"threads_started":1}
I20260812 01:13:06.451848 16598 tablet_service.cc:1467] Processing CreateTablet for tablet 702ef09898b04bc98dfd1aa8c43824e3 (DEFAULT_TABLE table=test-table [id=94359a2fe83245f99fca3420e0ec0486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:06.452207 17176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.445627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:40931 (local address 127.0.0.1:52104)
0812 01:13:06.446018 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.446028 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.446068 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.446319 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.446323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.446325 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:06.446415 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.446418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.448920 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.448920 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.450104 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.450106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.451925 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.451926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.452014 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.452019 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.452090 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.452135 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":66,"threads_started":1}
I20260812 01:13:06.452226 17042 tablet_service.cc:1467] Processing CreateTablet for tablet 702ef09898b04bc98dfd1aa8c43824e3 (DEFAULT_TABLE table=test-table [id=94359a2fe83245f99fca3420e0ec0486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:06.452383 16598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 702ef09898b04bc98dfd1aa8c43824e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:06.452608 17042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 702ef09898b04bc98dfd1aa8c43824e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:06.452649 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.445885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.17.130.2:40931)
0812 01:13:06.446118 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.446123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.446136 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:13:06.446182 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.446184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.446200 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.446291 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:06.446504 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.447684 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.450468 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.451817 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.451905 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.451998 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.452499 (+ 501us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.452552 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":106,"threads_started":1}
I20260812 01:13:06.453783 16821 tablet_service.cc:1467] Processing CreateTablet for tablet 702ef09898b04bc98dfd1aa8c43824e3 (DEFAULT_TABLE table=test-table [id=94359a2fe83245f99fca3420e0ec0486]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:06.454121 16821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 702ef09898b04bc98dfd1aa8c43824e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:06.455340 17185 tablet_bootstrap.cc:492] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: Bootstrap starting.
I20260812 01:13:06.456369 17186 tablet_bootstrap.cc:492] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Bootstrap starting.
I20260812 01:13:06.456442 17185 tablet_bootstrap.cc:654] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:06.456830 17185 log.cc:824] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:06.457146 17186 tablet_bootstrap.cc:654] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:06.457445 17191 tablet_bootstrap.cc:492] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Bootstrap starting.
I20260812 01:13:06.457479 17186 log.cc:824] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:06.457726 17185 tablet_bootstrap.cc:492] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: No bootstrap required, opened a new log
I20260812 01:13:06.457808 17185 ts_tablet_manager.cc:1397] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 01:13:06.458103 17191 tablet_bootstrap.cc:654] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:06.458252 17186 tablet_bootstrap.cc:492] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: No bootstrap required, opened a new log
I20260812 01:13:06.458334 17186 ts_tablet_manager.cc:1397] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:06.458444 17191 log.cc:824] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:06.459144 17191 tablet_bootstrap.cc:492] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: No bootstrap required, opened a new log
I20260812 01:13:06.459230 17191 ts_tablet_manager.cc:1397] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 01:13:06.460698 17185 raft_consensus.cc:348] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.460896 17185 raft_consensus.cc:374] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:06.460948 17185 raft_consensus.cc:729] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e1e90a9abd425fa8da22077f76ef8a, State: Initialized, Role: FOLLOWER
I20260812 01:13:06.461107 17186 raft_consensus.cc:348] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.461266 17186 raft_consensus.cc:374] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:06.461220 17185 consensus_queue.cc:260] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.461301 17186 raft_consensus.cc:729] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada99532b0c34126b29a04ee0c8709f5, State: Initialized, Role: FOLLOWER
I20260812 01:13:06.461433 17186 consensus_queue.cc:260] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.461530 17185 ts_tablet_manager.cc:1428] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260812 01:13:06.461684 16707 heartbeater.cc:503] Master 127.17.130.62:35311 was elected leader, sending a full tablet report...
I20260812 01:13:06.461700 17186 ts_tablet_manager.cc:1428] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260812 01:13:06.461833 17161 heartbeater.cc:503] Master 127.17.130.62:35311 was elected leader, sending a full tablet report...
I20260812 01:13:06.461813 17191 raft_consensus.cc:348] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.461982 17191 raft_consensus.cc:374] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:06.462033 17191 raft_consensus.cc:729] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a05fc67646c74811985f65f57dc8a6b6, State: Initialized, Role: FOLLOWER
I20260812 01:13:06.462147 17191 consensus_queue.cc:260] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.462458 17191 ts_tablet_manager.cc:1428] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:06.462502 16929 heartbeater.cc:503] Master 127.17.130.62:35311 was elected leader, sending a full tablet report...
W20260812 01:13:06.523511 16931 tablet.cc:2369] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:06.579908 17199 raft_consensus.cc:482] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:13:06.580138 17199 raft_consensus.cc:504] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.580626 17199 leader_election.cc:288] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e1e90a9abd425fa8da22077f76ef8a (127.17.130.1:45579), ada99532b0c34126b29a04ee0c8709f5 (127.17.130.3:37035)
I20260812 01:13:06.584168 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.580830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:37035 (local address 127.17.130.2:45153)
0812 01:13:06.580894 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.580982 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.581114 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.581359 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.581360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.581363 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:06.581470 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.581473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.582256 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.582257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.583124 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.583127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.583857 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.583858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.583940 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.583945 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.584017 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.584063 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 01:13:06.584296 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.580939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:45153 (local address 127.17.130.3:37035)
0812 01:13:06.581012 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.581014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.581086 (+ 72us) server_negotiation.cc:407] Connection header received
0812 01:13:06.581208 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.581210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.581221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.581280 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:06.581569 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.582093 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.583321 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.583767 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.583824 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.583832 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.584070 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.584108 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:13:06.584553 17200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.580865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:45579 (local address 127.17.130.2:59985)
0812 01:13:06.581104 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.581115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:06.581213 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:06.581499 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:06.581500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:06.581502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:06.581599 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.581601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.582430 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.582431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.583444 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:06.583447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.584268 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.584269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:06.584355 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.584359 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:06.584421 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 01:13:06.584464 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":149,"threads_started":1}
I20260812 01:13:06.584585 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.581005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:59985 (local address 127.17.130.1:45579)
0812 01:13:06.581105 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.581107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.581227 (+ 120us) server_negotiation.cc:407] Connection header received
0812 01:13:06.581361 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.581362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.581384 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.581479 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:06.581712 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.582320 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.583566 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.584083 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.584153 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.584160 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.584448 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.584514 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 01:13:06.584635 17072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ef09898b04bc98dfd1aa8c43824e3" candidate_uuid: "a05fc67646c74811985f65f57dc8a6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada99532b0c34126b29a04ee0c8709f5" is_pre_election: true
I20260812 01:13:06.584860 17072 raft_consensus.cc:2454] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a05fc67646c74811985f65f57dc8a6b6 in term 0.
I20260812 01:13:06.584875 16629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ef09898b04bc98dfd1aa8c43824e3" candidate_uuid: "a05fc67646c74811985f65f57dc8a6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e1e90a9abd425fa8da22077f76ef8a" is_pre_election: true
I20260812 01:13:06.585043 16629 raft_consensus.cc:2454] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a05fc67646c74811985f65f57dc8a6b6 in term 0.
I20260812 01:13:06.585183 16744 leader_election.cc:302] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a05fc67646c74811985f65f57dc8a6b6, ada99532b0c34126b29a04ee0c8709f5; no voters:
I20260812 01:13:06.585436 17199 raft_consensus.cc:2781] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:13:06.585518 17199 raft_consensus.cc:482] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:13:06.585567 17199 raft_consensus.cc:3037] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:06.586447 17199 raft_consensus.cc:504] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.586623 17199 leader_election.cc:288] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [CANDIDATE]: Term 1 election: Requested vote from peers 79e1e90a9abd425fa8da22077f76ef8a (127.17.130.1:45579), ada99532b0c34126b29a04ee0c8709f5 (127.17.130.3:37035)
I20260812 01:13:06.586850 17072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ef09898b04bc98dfd1aa8c43824e3" candidate_uuid: "a05fc67646c74811985f65f57dc8a6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ada99532b0c34126b29a04ee0c8709f5"
I20260812 01:13:06.586829 16629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ef09898b04bc98dfd1aa8c43824e3" candidate_uuid: "a05fc67646c74811985f65f57dc8a6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e1e90a9abd425fa8da22077f76ef8a"
I20260812 01:13:06.586932 17072 raft_consensus.cc:3037] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:06.586933 16629 raft_consensus.cc:3037] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:06.587904 16629 raft_consensus.cc:2454] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a05fc67646c74811985f65f57dc8a6b6 in term 1.
I20260812 01:13:06.587904 17072 raft_consensus.cc:2454] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a05fc67646c74811985f65f57dc8a6b6 in term 1.
I20260812 01:13:06.588141 16744 leader_election.cc:302] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a05fc67646c74811985f65f57dc8a6b6, ada99532b0c34126b29a04ee0c8709f5; no voters:
I20260812 01:13:06.588390 17199 raft_consensus.cc:2781] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:06.588600 17199 raft_consensus.cc:686] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 1 LEADER]: Becoming Leader. State: Replica: a05fc67646c74811985f65f57dc8a6b6, State: Running, Role: LEADER
I20260812 01:13:06.588711 17199 consensus_queue.cc:237] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } }
I20260812 01:13:06.589675 16400 catalog_manager.cc:5495] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a05fc67646c74811985f65f57dc8a6b6 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "a05fc67646c74811985f65f57dc8a6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a05fc67646c74811985f65f57dc8a6b6" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 40931 } health_report { overall_health: HEALTHY } } }
I20260812 01:13:06.601292 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:06.598973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52118 (local address 127.17.130.2:40931)
0812 01:13:06.599025 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:06.599027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:06.599037 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:06.599106 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:06.599107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:06.599125 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:06.599191 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:06.599320 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.599776 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.600493 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:06.600697 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:06.600753 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:06.600881 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:06.600969 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:06.601040 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:06.601160 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 01:13:06.601173 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:13:06.603441 16629 raft_consensus.cc:1261] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 1 FOLLOWER]: Refusing update from remote peer a05fc67646c74811985f65f57dc8a6b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:06.603449 17072 raft_consensus.cc:1261] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 1 FOLLOWER]: Refusing update from remote peer a05fc67646c74811985f65f57dc8a6b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:06.603825 17199 consensus_queue.cc:1035] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ada99532b0c34126b29a04ee0c8709f5" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 37035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:06.603995 17201 consensus_queue.cc:1035] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e1e90a9abd425fa8da22077f76ef8a" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 45579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:06.607376 17206 mvcc.cc:205] Tried to move back new op lower bound from 7317492476324577280 to 7317492476268027904. Current Snapshot: MvccSnapshot[applied={T|T < 7317492476324577280}]
I20260812 01:13:06.607746 17207 mvcc.cc:205] Tried to move back new op lower bound from 7317492476324577280 to 7317492476268027904. Current Snapshot: MvccSnapshot[applied={T|T < 7317492476324577280 or (T in {7317492476324577280})}]
I20260812 01:13:06.609678 16400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:06.609788 16400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:06.610814 16400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9a490db44f44cd59f909bb45e2baae8: Sending DeleteTablet for 3 replicas of tablet 702ef09898b04bc98dfd1aa8c43824e3
I20260812 01:13:06.611193 16821 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ef09898b04bc98dfd1aa8c43824e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:06 UTC) from {username='slave'} at 127.0.0.1:52104
I20260812 01:13:06.611249 16598 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ef09898b04bc98dfd1aa8c43824e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:06 UTC) from {username='slave'} at 127.0.0.1:37600
I20260812 01:13:06.611258 17042 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ef09898b04bc98dfd1aa8c43824e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:06 UTC) from {username='slave'} at 127.0.0.1:49468
I20260812 01:13:06.611579 17218 tablet_replica.cc:330] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a: stopping tablet replica
I20260812 01:13:06.611696 17218 raft_consensus.cc:2229] T 702ef09898b04bc98dfd1aa8c43824e3 P 79e1e90a9abd425fa8da22077f76ef8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:06.611826 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 16491
I20260812 01:13:06.656891 17217 tablet_replica.cc:330] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: stopping tablet replica
I20260812 01:13:06.657131 17217 raft_consensus.cc:2229] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:13:06.657312 17217 raft_consensus.cc:2258] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:06.657541 17220 tablet_replica.cc:330] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: stopping tablet replica
I20260812 01:13:06.657668 17220 raft_consensus.cc:2229] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:06.657783 17220 raft_consensus.cc:2258] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:06.658385 17220 ts_tablet_manager.cc:1905] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:06.660168 17220 ts_tablet_manager.cc:1918] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:06.660259 17220 log.cc:1196] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/702ef09898b04bc98dfd1aa8c43824e3
I20260812 01:13:06.660524 17220 ts_tablet_manager.cc:1939] T 702ef09898b04bc98dfd1aa8c43824e3 P ada99532b0c34126b29a04ee0c8709f5: Deleting consensus metadata
I20260812 01:13:06.661154 16375 catalog_manager.cc:4841] TS ada99532b0c34126b29a04ee0c8709f5 (127.17.130.3:37035): tablet 702ef09898b04bc98dfd1aa8c43824e3 (table test-table [id=94359a2fe83245f99fca3420e0ec0486]) successfully deleted
I20260812 01:13:06.661583 17217 ts_tablet_manager.cc:1905] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 01:13:06.663535 16376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:45579 (error 108)
I20260812 01:13:06.663743 17217 ts_tablet_manager.cc:1918] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:06.663841 17217 log.cc:1196] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/702ef09898b04bc98dfd1aa8c43824e3
I20260812 01:13:06.664228 17217 ts_tablet_manager.cc:1939] T 702ef09898b04bc98dfd1aa8c43824e3 P a05fc67646c74811985f65f57dc8a6b6: Deleting consensus metadata
W20260812 01:13:06.664276 17176 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:06.664107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:45579 (local address 127.0.0.1:37616)
0812 01:13:06.664178 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:06.664229 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:45579: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 01:13:06.664405 16376 catalog_manager.cc:4584] TS 79e1e90a9abd425fa8da22077f76ef8a (127.17.130.1:45579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 702ef09898b04bc98dfd1aa8c43824e3: Network error: Client connection negotiation failed: client connection to 127.17.130.1:45579: connect: Connection reset by peer (error 104)
I20260812 01:13:06.664714 16374 catalog_manager.cc:4841] TS a05fc67646c74811985f65f57dc8a6b6 (127.17.130.2:40931): tablet 702ef09898b04bc98dfd1aa8c43824e3 (table test-table [id=94359a2fe83245f99fca3420e0ec0486]) successfully deleted
I20260812 01:13:06.664819 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 16712
I20260812 01:13:06.674150 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 16936
I20260812 01:13:06.684877 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 16349
2026-08-12T01:13:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (928 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:06 dist-test-slave-tfpf krb5kdc[17227](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:06 dist-test-slave-tfpf krb5kdc[17227](info): set up 2 sockets
Aug 12 01:13:06 dist-test-slave-tfpf krb5kdc[17227](info): commencing operation
krb5kdc: starting...
W20260812 01:13:08.011071 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:08.152185 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:39613
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:39613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:08.248888 17247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:08.249225 17247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:08.249277 17247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:08.250578 17247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:08.250658 17247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:08.250685 17247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:08.250716 17247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:08.250747 17247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:08.252417 17247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:39613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:39613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:08.252729 17247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:08.252996 17247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:08.259388 17258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.259451 17262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.260180 17256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:08.261154 17247 server_base.cc:1034] running on GCE node
I20260812 01:13:08.261504 17247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:08.261963 17247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:08.263234 17247 hybrid_clock.cc:648] HybridClock initialized: now 1786497188263162 us; error 129 us; skew 500 ppm
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:08.266745 17247 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:08.267215 17247 webserver.cc:460] Webserver started at http://127.17.130.62:44281/ using document root <none> and password file <none>
I20260812 01:13:08.267467 17247 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:08.267519 17247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:08.267663 17247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:08.268853 17247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "3efd8d5a080e4d60aa71d0d2d70c6190"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "96f8bd63d1f096974ff9cabb545ae1c0"
server_key_iv: "d8ae58ea76e194c4256eef4d1a40b4a5"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.269353 17247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "3efd8d5a080e4d60aa71d0d2d70c6190"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "96f8bd63d1f096974ff9cabb545ae1c0"
server_key_iv: "d8ae58ea76e194c4256eef4d1a40b4a5"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.270943 17247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:08.271975 17274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.272339 17247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:08.272500 17247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "3efd8d5a080e4d60aa71d0d2d70c6190"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "96f8bd63d1f096974ff9cabb545ae1c0"
server_key_iv: "d8ae58ea76e194c4256eef4d1a40b4a5"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.272594 17247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:08.287782 17247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:08.290484 17247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:08.290702 17247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:08.295666 17247 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:39613
I20260812 01:13:08.295713 17364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:39613 every 8 connection(s)
I20260812 01:13:08.296222 17247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:08.297510 17365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:08.299569 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 17247
I20260812 01:13:08.299710 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:08.299921 17928 external_mini_cluster.cc:1428] Setting key bcd29749fbdabcbd65d3e0917e70cbea
I20260812 01:13:08.300033 17365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190: Bootstrap starting.
I20260812 01:13:08.300868 17365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:08.301254 17365 log.cc:824] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:08.302155 17365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190: No bootstrap required, opened a new log
I20260812 01:13:08.304651 17365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39613 } }
I20260812 01:13:08.304831 17365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:08.304854 17365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3efd8d5a080e4d60aa71d0d2d70c6190, State: Initialized, Role: FOLLOWER
I20260812 01:13:08.304997 17365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39613 } }
I20260812 01:13:08.305127 17365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:08.305177 17365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:08.305229 17365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:08.306077 17365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39613 } }
I20260812 01:13:08.306216 17365 leader_election.cc:302] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3efd8d5a080e4d60aa71d0d2d70c6190; no voters:
I20260812 01:13:08.306413 17365 leader_election.cc:288] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:08.306486 17380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:08.306671 17380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [term 1 LEADER]: Becoming Leader. State: Replica: 3efd8d5a080e4d60aa71d0d2d70c6190, State: Running, Role: LEADER
I20260812 01:13:08.306766 17365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:08.306780 17380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39613 } }
I20260812 01:13:08.307159 17380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3efd8d5a080e4d60aa71d0d2d70c6190. Latest consensus state: current_term: 1 leader_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39613 } } }
I20260812 01:13:08.307257 17380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:08.307170 17382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3efd8d5a080e4d60aa71d0d2d70c6190" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 39613 } } }
I20260812 01:13:08.307493 17387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:08.307498 17382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190 [sys.catalog]: This master's current role is: LEADER
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:08.308230 17387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:08.313197 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.300585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.17.130.62:39613)
0812 01:13:08.300773 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.300781 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.300806 (+ 25us) server_negotiation.cc:407] Connection header received
0812 01:13:08.301394 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.301409 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.301475 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.301646 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:08.302436 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.303477 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.304648 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.305321 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.305424 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.308694 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:08.308699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:08.308699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:08.308715 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:08.311310 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.311756 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.311759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.311760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.311869 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.312200 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.312201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.312202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.312527 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:08.312671 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.312780 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.312926 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
I20260812 01:13:08.314411 17387 catalog_manager.cc:1347] Generated new cluster ID: 98e8628140dc49779ae7845c48de9112
I20260812 01:13:08.314473 17387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:08.343809 17387 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:08.344545 17387 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:08.352536 17387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190: Generated new TSK 0
I20260812 01:13:08.352798 17387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.17.130.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
Entry for principal kudu/127.17.130.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab.
I20260812 01:13:08.391743 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39613
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:08.486114 17417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:08.486361 17417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:08.486387 17417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:08.487561 17417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:08.487605 17417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:08.487634 17417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:13:08.489387 17417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.1.keytab
--principal=kudu/127.17.130.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:08.489631 17417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:08.489831 17417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:08.495260 17425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.495237 17428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.495265 17423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:08.496173 17417 server_base.cc:1034] running on GCE node
I20260812 01:13:08.496393 17417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:08.496640 17417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:08.497948 17417 hybrid_clock.cc:648] HybridClock initialized: now 1786497188497875 us; error 97 us; skew 500 ppm
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:08.501958 17417 init.cc:377] Logged in from keytab as kudu/127.17.130.1@KRBTEST.COM (short username kudu)
I20260812 01:13:08.502750 17417 webserver.cc:460] Webserver started at http://127.17.130.1:44851/ using document root <none> and password file <none>
I20260812 01:13:08.503104 17417 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:08.503168 17417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:08.503341 17417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:08.504693 17417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "3bbd0ccb46a4490ab903198e56594c1f"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "d5bd5e21a26d19c174a5614ce6e2f65e"
server_key_iv: "7dd617d6d205aee2e46b59c46ad6791d"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.505491 17417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "3bbd0ccb46a4490ab903198e56594c1f"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "d5bd5e21a26d19c174a5614ce6e2f65e"
server_key_iv: "7dd617d6d205aee2e46b59c46ad6791d"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.507332 17417 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:08.508383 17438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.508556 17417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:08.508679 17417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "3bbd0ccb46a4490ab903198e56594c1f"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "d5bd5e21a26d19c174a5614ce6e2f65e"
server_key_iv: "7dd617d6d205aee2e46b59c46ad6791d"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.508795 17417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:08.517501 17417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:08.519972 17417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:08.520164 17417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:08.520419 17417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:08.520761 17417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:08.520817 17417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.520864 17417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:08.520895 17417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.528434 17417 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:37903
I20260812 01:13:08.528514 17639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:37903 every 8 connection(s)
I20260812 01:13:08.528926 17417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.1@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:08.539054 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 17417
I20260812 01:13:08.539206 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:13:08.539281 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.529916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:50475 (local address 127.17.130.62:39613)
0812 01:13:08.530019 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.530022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.530209 (+ 187us) server_negotiation.cc:407] Connection header received
0812 01:13:08.530767 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.530768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.530779 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.530849 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:08.531919 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.532446 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.533513 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.533727 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.533797 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.536302 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:08.536304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:08.536304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:08.536311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:08.537717 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.538149 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.538150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.538151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.538201 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.538466 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.538467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.538467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.538569 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:08.538623 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.539045 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.539132 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 01:13:08.539433 17928 external_mini_cluster.cc:1428] Setting key ff97740b884733eb5e8f4b66ccc8dc74
I20260812 01:13:08.539572 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.529823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39613 (local address 127.17.130.1:50475)
0812 01:13:08.530044 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:08.530088 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:08.530633 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:08.530945 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:08.530949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:08.531590 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:08.531801 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.531807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.532568 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.532569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.533408 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.533411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.533849 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.533850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.533972 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.534616 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:08.534625 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:08.536210 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:08.537809 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:08.537816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:08.537818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:08.538059 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:08.538283 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:08.538284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:08.538284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:08.538401 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:08.538638 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:08.538645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:08.538772 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 01:13:08.538958 (+ 186us) client_negotiation.cc:240] Negotiation successful
0812 01:13:08.539174 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":115,"threads_started":1}
I20260812 01:13:08.540426 17640 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39613
I20260812 01:13:08.540561 17640 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:08.540784 17640 heartbeater.cc:511] Master 127.17.130.62:39613 requested a full tablet report, sending...
I20260812 01:13:08.541626 17302 ts_manager.cc:194] Registered new tserver with Master: 3bbd0ccb46a4490ab903198e56594c1f (127.17.130.1:37903)
I20260812 01:13:08.542275 17302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.1@KRBTEST.COM'} at 127.17.130.1:50475
WARNING: no policy specified for kudu/127.17.130.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
Entry for principal kudu/127.17.130.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab.
I20260812 01:13:08.573647 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39613
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:08.665448 17647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:08.665660 17647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:08.665686 17647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:08.666761 17647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:08.666800 17647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:08.666826 17647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:13:08.668479 17647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.2.keytab
--principal=kudu/127.17.130.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:08.668716 17647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:08.668908 17647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:08.674396 17660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.674417 17657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.674378 17662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:08.674629 17647 server_base.cc:1034] running on GCE node
I20260812 01:13:08.675024 17647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:08.675315 17647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:08.676509 17647 hybrid_clock.cc:648] HybridClock initialized: now 1786497188676476 us; error 52 us; skew 500 ppm
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:08.679240 17647 init.cc:377] Logged in from keytab as kudu/127.17.130.2@KRBTEST.COM (short username kudu)
I20260812 01:13:08.679733 17647 webserver.cc:460] Webserver started at http://127.17.130.2:35749/ using document root <none> and password file <none>
I20260812 01:13:08.679991 17647 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:08.680073 17647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:08.680212 17647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:08.681422 17647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "6ffe831afc734c5293a0d7882eb267b0"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "000916969852fb026e7780f966ffbee6"
server_key_iv: "233b502995cafa39866741f9e50a7aed"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.681798 17647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "6ffe831afc734c5293a0d7882eb267b0"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "000916969852fb026e7780f966ffbee6"
server_key_iv: "233b502995cafa39866741f9e50a7aed"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.683251 17647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:13:08.684226 17671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.684459 17647 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:08.684558 17647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "6ffe831afc734c5293a0d7882eb267b0"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "000916969852fb026e7780f966ffbee6"
server_key_iv: "233b502995cafa39866741f9e50a7aed"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.684612 17647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:08.694981 17647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:08.697436 17647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:08.697566 17647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:08.697786 17647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:08.698141 17647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:08.698179 17647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.698201 17647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:08.698244 17647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.704071 17647 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:46317
I20260812 01:13:08.704198 17868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:46317 every 8 connection(s)
I20260812 01:13:08.704504 17647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:13:08.710685 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 17647
I20260812 01:13:08.710831 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:13:08.711061 17928 external_mini_cluster.cc:1428] Setting key 2a233cbcb278d128445daad34cd594cc
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.2@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:08.716322 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.705469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:55363 (local address 127.17.130.62:39613)
0812 01:13:08.705537 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.705539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.705783 (+ 244us) server_negotiation.cc:407] Connection header received
0812 01:13:08.706386 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.706387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.706447 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.706528 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:08.707712 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.708278 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.709442 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.709642 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.709700 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.712608 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:08.712609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:08.712610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:08.712618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:08.714508 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.715116 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.715117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.715117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.715178 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.715529 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.715530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.715530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.715654 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:08.715720 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.716088 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.716184 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 01:13:08.716356 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.705412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39613 (local address 127.17.130.2:55363)
0812 01:13:08.705600 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:08.705647 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:08.706258 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:08.706615 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:08.706619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:08.707389 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:08.707614 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.707620 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.708366 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.708367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.709357 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.709360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.709723 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.709724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.709832 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.710483 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:08.710495 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:08.712506 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:08.714605 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:08.714614 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:08.714618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:08.715010 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:08.715261 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:08.715262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:08.715263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:08.715454 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:08.715736 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:08.715743 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:08.715896 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 01:13:08.716039 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 01:13:08.716180 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
I20260812 01:13:08.717293 17869 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39613
I20260812 01:13:08.717419 17869 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:08.717639 17869 heartbeater.cc:511] Master 127.17.130.62:39613 requested a full tablet report, sending...
I20260812 01:13:08.718147 17302 ts_manager.cc:194] Registered new tserver with Master: 6ffe831afc734c5293a0d7882eb267b0 (127.17.130.2:46317)
I20260812 01:13:08.718545 17302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.2@KRBTEST.COM'} at 127.17.130.2:55363
WARNING: no policy specified for kudu/127.17.130.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
Entry for principal kudu/127.17.130.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab.
I20260812 01:13:08.743716 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39613
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:08.833457 17875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:08.833702 17875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:08.833729 17875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:08.834916 17875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:08.834960 17875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:08.834990 17875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:13:08.836449 17875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.3.keytab
--principal=kudu/127.17.130.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:39613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:08.836686 17875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:08.836892 17875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:08.842361 17886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:08.842352 17893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:08.842595 17875 server_base.cc:1034] running on GCE node
W20260812 01:13:08.842357 17889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:08.842905 17875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:08.843295 17875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:08.844509 17875 hybrid_clock.cc:648] HybridClock initialized: now 1786497188844500 us; error 51 us; skew 500 ppm
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:08.847374 17875 init.cc:377] Logged in from keytab as kudu/127.17.130.3@KRBTEST.COM (short username kudu)
I20260812 01:13:08.847909 17875 webserver.cc:460] Webserver started at http://127.17.130.3:39917/ using document root <none> and password file <none>
I20260812 01:13:08.848196 17875 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:08.848270 17875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:08.848429 17875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:08.849730 17875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "c2da62f246f08bb5dfdddc780caa5c48"
server_key_iv: "6ed10ee58cfbec51b28c4d8bdeb3fdaf"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.850133 17875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "c2da62f246f08bb5dfdddc780caa5c48"
server_key_iv: "6ed10ee58cfbec51b28c4d8bdeb3fdaf"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.851580 17875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:13:08.852409 17909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.852643 17875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:08.852751 17875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7"
format_stamp: "Formatted at 2026-08-12 01:13:08 on dist-test-slave-tfpf"
server_key: "c2da62f246f08bb5dfdddc780caa5c48"
server_key_iv: "6ed10ee58cfbec51b28c4d8bdeb3fdaf"
server_key_version: "encryptionkey@0"
I20260812 01:13:08.852803 17875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:08.877624 17875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:08.880218 17875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:08.880344 17875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:08.880563 17875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:08.880877 17875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:08.880910 17875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.880942 17875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:08.880955 17875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:08.887637 17875 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:36809
I20260812 01:13:08.887699 18118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:36809 every 8 connection(s)
I20260812 01:13:08.888070 17875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:13:08.890349 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 17875
I20260812 01:13:08.890444 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:13:08.890661 17928 external_mini_cluster.cc:1428] Setting key e8f048d86cdaa19ff5f7f65226807662
Aug 12 01:13:08 dist-test-slave-tfpf krb5kdc[17227](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497188, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.3@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:08.901247 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.889048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:39613 (local address 127.17.130.3:39365)
0812 01:13:08.889365 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:08.889405 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:08.889920 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:08.890217 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:08.890220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:08.890966 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:08.891195 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.891201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.892184 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.892186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.893447 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.893452 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.893914 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.893916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.894052 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.894831 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:08.894841 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:08.896692 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:08.898951 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:08.898961 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:08.898963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:08.899238 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:08.899542 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:08.899543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:08.899543 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:08.899681 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:08.900506 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:08.900513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:08.900675 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 01:13:08.900785 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 01:13:08.900954 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":126,"threads_started":1}
I20260812 01:13:08.901274 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.889163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:39365 (local address 127.17.130.62:39613)
0812 01:13:08.889252 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.889255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.889495 (+ 240us) server_negotiation.cc:407] Connection header received
0812 01:13:08.890042 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.890043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.890055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.890129 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:08.891266 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.891992 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.893571 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.893769 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.893864 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.896819 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:08.896820 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:08.896821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:08.896830 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:08.898818 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.899339 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.899340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.899340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.899406 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.899789 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.899790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.899791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.900396 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:08.900475 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.900927 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.901152 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 01:13:08.902141 18119 heartbeater.cc:347] Connected to a master server at 127.17.130.62:39613
I20260812 01:13:08.902279 18119 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:08.902493 18119 heartbeater.cc:511] Master 127.17.130.62:39613 requested a full tablet report, sending...
I20260812 01:13:08.903047 17302 ts_manager.cc:194] Registered new tserver with Master: 0a8599cfbc9b4cca9f022399a8fb9ca7 (127.17.130.3:36809)
I20260812 01:13:08.903739 17302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.17.130.3@KRBTEST.COM'} at 127.17.130.3:39365
I20260812 01:13:08.913559 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:13:08.923280 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.915140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.17.130.62:39613)
0812 01:13:08.915233 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.915235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.915262 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:13:08.915335 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.915336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.915346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.915413 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:08.916181 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.916700 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.917809 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.918002 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.918068 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.919643 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:08.919645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:08.919645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:08.919654 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:08.921404 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.922085 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.922087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.922088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.922156 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:08.922439 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:08.922441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:08.922441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:08.922633 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:08.922801 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.922853 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.923198 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 01:13:08.925840 17302 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:13:08.927619 17302 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:13:08.935541 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.931077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:37903 (local address 127.0.0.1:60822)
0812 01:13:08.931316 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:08.931364 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:08.931445 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:08.931948 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:08.931951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:08.931953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:08.932046 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.932049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.933024 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.933025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.934292 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.934294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.935206 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.935207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.935276 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.935297 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 01:13:08.935342 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 01:13:08.935400 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":52,"thread_start_us":104,"threads_started":1}
I20260812 01:13:08.935570 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.931562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.17.130.3:36809)
0812 01:13:08.931859 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.931864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.931873 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:08.931909 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.931912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.931926 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.932034 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:08.932134 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.932847 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.934239 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.935037 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.935133 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.935210 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.935429 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.935483 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
I20260812 01:13:08.935570 18149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.931322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36809 (local address 127.0.0.1:42334)
0812 01:13:08.931577 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:08.931586 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:08.931629 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:08.932037 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:08.932038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:08.932039 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:08.932073 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.932074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.932952 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.932953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.934147 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.934150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.935167 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.935168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.935271 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.935287 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:13:08.935383 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 01:13:08.935445 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":17,"thread_start_us":94,"threads_started":1}
I20260812 01:13:08.936116 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.931249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.17.130.1:37903)
0812 01:13:08.931753 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.931757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.931768 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:13:08.931813 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.931815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.931832 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.932145 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:08.932221 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.932883 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.934379 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.935107 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.935878 (+ 771us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.935950 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.935979 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.936033 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":105,"threads_started":1}
I20260812 01:13:08.936563 18005 tablet_service.cc:1467] Processing CreateTablet for tablet 6affaf75288f4bb8a2f56331085420f2 (DEFAULT_TABLE table=test-table [id=d313f697a11d4b5b95897c4264d821a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:08.936826 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.931077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:46317 (local address 127.0.0.1:60058)
0812 01:13:08.931533 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:08.931544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:08.931654 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:08.931972 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:08.931973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:08.931975 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:08.932054 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.932056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.934265 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.934266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.935168 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:08.935169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.936548 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.936549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:08.936618 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.936623 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:08.936702 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 01:13:08.936745 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":7,"thread_start_us":77,"threads_started":1}
I20260812 01:13:08.936851 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:08.931344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.17.130.2:46317)
0812 01:13:08.931512 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:08.931517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:08.931723 (+ 206us) server_negotiation.cc:407] Connection header received
0812 01:13:08.931774 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:08.931776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:08.931795 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:08.931873 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:08.932951 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.934136 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.935263 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:08.936446 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:08.936517 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:08.936553 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:08.936731 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 01:13:08.936775 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260812 01:13:08.937057 18005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6affaf75288f4bb8a2f56331085420f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:08.937012 17517 tablet_service.cc:1467] Processing CreateTablet for tablet 6affaf75288f4bb8a2f56331085420f2 (DEFAULT_TABLE table=test-table [id=d313f697a11d4b5b95897c4264d821a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:08.937407 17517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6affaf75288f4bb8a2f56331085420f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:08.940179 18153 tablet_bootstrap.cc:492] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: Bootstrap starting.
I20260812 01:13:08.940594 17752 tablet_service.cc:1467] Processing CreateTablet for tablet 6affaf75288f4bb8a2f56331085420f2 (DEFAULT_TABLE table=test-table [id=d313f697a11d4b5b95897c4264d821a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:08.940877 18153 tablet_bootstrap.cc:654] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:08.940941 17752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6affaf75288f4bb8a2f56331085420f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:08.941241 18154 tablet_bootstrap.cc:492] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Bootstrap starting.
I20260812 01:13:08.941517 18153 log.cc:824] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:08.941937 18154 tablet_bootstrap.cc:654] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:08.942474 18153 tablet_bootstrap.cc:492] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: No bootstrap required, opened a new log
I20260812 01:13:08.942517 18154 log.cc:824] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:08.942663 18153 ts_tablet_manager.cc:1397] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 01:13:08.943318 18154 tablet_bootstrap.cc:492] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: No bootstrap required, opened a new log
I20260812 01:13:08.943400 18154 ts_tablet_manager.cc:1397] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:08.944299 18153 raft_consensus.cc:348] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:08.944540 18153 raft_consensus.cc:374] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:08.944586 18153 raft_consensus.cc:729] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bbd0ccb46a4490ab903198e56594c1f, State: Initialized, Role: FOLLOWER
I20260812 01:13:08.944496 18159 tablet_bootstrap.cc:492] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Bootstrap starting.
I20260812 01:13:08.944733 18153 consensus_queue.cc:260] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:08.945115 17640 heartbeater.cc:503] Master 127.17.130.62:39613 was elected leader, sending a full tablet report...
I20260812 01:13:08.945104 18153 ts_tablet_manager.cc:1428] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:08.945415 18159 tablet_bootstrap.cc:654] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:08.945650 18154 raft_consensus.cc:348] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:08.945813 18154 raft_consensus.cc:374] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:08.945858 18154 raft_consensus.cc:729] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8599cfbc9b4cca9f022399a8fb9ca7, State: Initialized, Role: FOLLOWER
I20260812 01:13:08.945976 18154 consensus_queue.cc:260] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:08.946048 18159 log.cc:824] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:08.946295 18154 ts_tablet_manager.cc:1428] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:08.946346 18119 heartbeater.cc:503] Master 127.17.130.62:39613 was elected leader, sending a full tablet report...
I20260812 01:13:08.946861 18159 tablet_bootstrap.cc:492] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: No bootstrap required, opened a new log
I20260812 01:13:08.946938 18159 ts_tablet_manager.cc:1397] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 01:13:08.948499 18159 raft_consensus.cc:348] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:08.948673 18159 raft_consensus.cc:374] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:08.948722 18159 raft_consensus.cc:729] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffe831afc734c5293a0d7882eb267b0, State: Initialized, Role: FOLLOWER
I20260812 01:13:08.948848 18159 consensus_queue.cc:260] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:08.949206 18159 ts_tablet_manager.cc:1428] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 01:13:08.949254 17869 heartbeater.cc:503] Master 127.17.130.62:39613 was elected leader, sending a full tablet report...
W20260812 01:13:08.955168 17871 tablet.cc:2369] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 01:13:09.029824 17641 tablet.cc:2369] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:09.107452 18160 raft_consensus.cc:482] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:13:09.107654 18160 raft_consensus.cc:504] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:09.108090 18160 leader_election.cc:288] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ffe831afc734c5293a0d7882eb267b0 (127.17.130.2:46317), 0a8599cfbc9b4cca9f022399a8fb9ca7 (127.17.130.3:36809)
I20260812 01:13:09.112479 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.108450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:35055 (local address 127.17.130.2:46317)
0812 01:13:09.108528 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.108530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.108574 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:13:09.108634 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.108635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.108645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.108713 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:09.108918 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.109698 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.111432 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.112070 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.112147 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.112155 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.112359 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.112403 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 01:13:09.112479 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.108535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:39653 (local address 127.17.130.3:36809)
0812 01:13:09.108580 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.108582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.108602 (+ 20us) server_negotiation.cc:407] Connection header received
0812 01:13:09.108634 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.108635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.108645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.108713 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:09.108922 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.109702 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.111432 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.112046 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.112139 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.112150 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.112359 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.112403 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:13:09.112571 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.108319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:36809 (local address 127.17.130.1:39653)
0812 01:13:09.108465 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.108479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.108555 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.108738 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.108739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.108741 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:09.108839 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.108840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.109812 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.109813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.111291 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.111294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.112160 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.112161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.112257 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.112264 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.112338 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.112393 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":76,"thread_start_us":94,"threads_started":1}
I20260812 01:13:09.112704 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.108323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:46317 (local address 127.17.130.1:35055)
0812 01:13:09.108515 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.108525 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.108577 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.108739 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.108740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.108741 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:09.108833 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.108836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.109811 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.109813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.111291 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.111294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.112158 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.112160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.112257 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.112265 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.112338 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.112393 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":145,"thread_start_us":73,"threads_started":1}
I20260812 01:13:09.113123 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6affaf75288f4bb8a2f56331085420f2" candidate_uuid: "3bbd0ccb46a4490ab903198e56594c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ffe831afc734c5293a0d7882eb267b0" is_pre_election: true
I20260812 01:13:09.113240 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6affaf75288f4bb8a2f56331085420f2" candidate_uuid: "3bbd0ccb46a4490ab903198e56594c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" is_pre_election: true
I20260812 01:13:09.113335 17794 raft_consensus.cc:2454] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bbd0ccb46a4490ab903198e56594c1f in term 0.
I20260812 01:13:09.113394 18045 raft_consensus.cc:2454] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bbd0ccb46a4490ab903198e56594c1f in term 0.
I20260812 01:13:09.113636 17442 leader_election.cc:302] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bbd0ccb46a4490ab903198e56594c1f, 6ffe831afc734c5293a0d7882eb267b0; no voters:
I20260812 01:13:09.113821 18160 raft_consensus.cc:2781] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:13:09.113914 18160 raft_consensus.cc:482] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:13:09.113938 18160 raft_consensus.cc:3037] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:09.114830 18160 raft_consensus.cc:504] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:09.114985 18160 leader_election.cc:288] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [CANDIDATE]: Term 1 election: Requested vote from peers 6ffe831afc734c5293a0d7882eb267b0 (127.17.130.2:46317), 0a8599cfbc9b4cca9f022399a8fb9ca7 (127.17.130.3:36809)
I20260812 01:13:09.115167 17794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6affaf75288f4bb8a2f56331085420f2" candidate_uuid: "3bbd0ccb46a4490ab903198e56594c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ffe831afc734c5293a0d7882eb267b0"
I20260812 01:13:09.115168 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6affaf75288f4bb8a2f56331085420f2" candidate_uuid: "3bbd0ccb46a4490ab903198e56594c1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7"
I20260812 01:13:09.115249 17794 raft_consensus.cc:3037] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:09.115279 18045 raft_consensus.cc:3037] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:09.116171 18045 raft_consensus.cc:2454] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bbd0ccb46a4490ab903198e56594c1f in term 1.
I20260812 01:13:09.116173 17794 raft_consensus.cc:2454] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bbd0ccb46a4490ab903198e56594c1f in term 1.
I20260812 01:13:09.116377 17442 leader_election.cc:302] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bbd0ccb46a4490ab903198e56594c1f, 6ffe831afc734c5293a0d7882eb267b0; no voters:
I20260812 01:13:09.116475 18160 raft_consensus.cc:2781] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:09.116631 18160 raft_consensus.cc:686] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 1 LEADER]: Becoming Leader. State: Replica: 3bbd0ccb46a4490ab903198e56594c1f, State: Running, Role: LEADER
I20260812 01:13:09.116720 18160 consensus_queue.cc:237] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } }
I20260812 01:13:09.117792 17302 catalog_manager.cc:5495] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bbd0ccb46a4490ab903198e56594c1f (127.17.130.1). New cstate: current_term: 1 leader_uuid: "3bbd0ccb46a4490ab903198e56594c1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bbd0ccb46a4490ab903198e56594c1f" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 37903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 } health_report { overall_health: UNKNOWN } } }
W20260812 01:13:09.138759 18127 tablet.cc:2369] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:09.163228 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.160739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60838 (local address 127.17.130.1:37903)
0812 01:13:09.160816 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.160819 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.160827 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:13:09.160882 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.160883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.160903 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.160979 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:09.161153 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.161648 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.162409 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.162612 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.162691 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.162872 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:09.162978 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:09.163077 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.163097 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.163118 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 01:13:09.165277 17794 raft_consensus.cc:1261] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 1 FOLLOWER]: Refusing update from remote peer 3bbd0ccb46a4490ab903198e56594c1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:09.165374 18045 raft_consensus.cc:1261] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 1 FOLLOWER]: Refusing update from remote peer 3bbd0ccb46a4490ab903198e56594c1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:09.165751 18170 consensus_queue.cc:1035] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ffe831afc734c5293a0d7882eb267b0" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 46317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:09.165907 18160 consensus_queue.cc:1035] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a8599cfbc9b4cca9f022399a8fb9ca7" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 36809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:09.168838 18177 mvcc.cc:205] Tried to move back new op lower bound from 7317492486817775616 to 7317492486622879744. Current Snapshot: MvccSnapshot[applied={T|T < 7317492486817775616}]
I20260812 01:13:09.168987 18176 mvcc.cc:205] Tried to move back new op lower bound from 7317492486817775616 to 7317492486622879744. Current Snapshot: MvccSnapshot[applied={T|T < 7317492486817775616}]
I20260812 01:13:09.171550 17303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:09.171681 17303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:09.172535 17303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3efd8d5a080e4d60aa71d0d2d70c6190: Sending DeleteTablet for 3 replicas of tablet 6affaf75288f4bb8a2f56331085420f2
I20260812 01:13:09.172937 17752 tablet_service.cc:1514] Processing DeleteTablet for tablet 6affaf75288f4bb8a2f56331085420f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:09 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:60058
I20260812 01:13:09.172937 18005 tablet_service.cc:1514] Processing DeleteTablet for tablet 6affaf75288f4bb8a2f56331085420f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:09 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:42334
I20260812 01:13:09.173239 17517 tablet_service.cc:1514] Processing DeleteTablet for tablet 6affaf75288f4bb8a2f56331085420f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:09 UTC) from {username='kudu', principal='kudu/127.17.130.62@KRBTEST.COM'} at 127.0.0.1:60822
I20260812 01:13:09.173372 18190 tablet_replica.cc:330] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: stopping tablet replica
I20260812 01:13:09.173487 18190 raft_consensus.cc:2229] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:09.173486 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 17417
I20260812 01:13:09.173496 18191 tablet_replica.cc:330] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f: stopping tablet replica
I20260812 01:13:09.173606 18191 raft_consensus.cc:2229] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:13:09.173738 18191 raft_consensus.cc:2258] T 6affaf75288f4bb8a2f56331085420f2 P 3bbd0ccb46a4490ab903198e56594c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:09.174045 18189 tablet_replica.cc:330] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: stopping tablet replica
I20260812 01:13:09.174152 18189 raft_consensus.cc:2229] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:09.174256 18189 raft_consensus.cc:2258] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:09.174392 18190 raft_consensus.cc:2258] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:09.177354 18190 ts_tablet_manager.cc:1905] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:09.177488 18189 ts_tablet_manager.cc:1905] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:09.179620 18189 ts_tablet_manager.cc:1918] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:09.179704 18189 log.cc:1196] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/6affaf75288f4bb8a2f56331085420f2
I20260812 01:13:09.179965 18189 ts_tablet_manager.cc:1939] T 6affaf75288f4bb8a2f56331085420f2 P 6ffe831afc734c5293a0d7882eb267b0: Deleting consensus metadata
I20260812 01:13:09.180492 17277 catalog_manager.cc:4841] TS 6ffe831afc734c5293a0d7882eb267b0 (127.17.130.2:46317): tablet 6affaf75288f4bb8a2f56331085420f2 (table test-table [id=d313f697a11d4b5b95897c4264d821a1]) successfully deleted
I20260812 01:13:09.180737 18190 ts_tablet_manager.cc:1918] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:09.180819 18190 log.cc:1196] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/6affaf75288f4bb8a2f56331085420f2
I20260812 01:13:09.181174 18190 ts_tablet_manager.cc:1939] T 6affaf75288f4bb8a2f56331085420f2 P 0a8599cfbc9b4cca9f022399a8fb9ca7: Deleting consensus metadata
I20260812 01:13:09.181658 17278 catalog_manager.cc:4841] TS 0a8599cfbc9b4cca9f022399a8fb9ca7 (127.17.130.3:36809): tablet 6affaf75288f4bb8a2f56331085420f2 (table test-table [id=d313f697a11d4b5b95897c4264d821a1]) successfully deleted
W20260812 01:13:09.185766 17278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:37903 (error 108)
I20260812 01:13:09.186036 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 17647
I20260812 01:13:09.186866 18144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.186255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:37903 (local address 127.0.0.1:60852)
0812 01:13:09.186754 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.186821 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:37903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476}
W20260812 01:13:09.187003 17278 catalog_manager.cc:4584] TS 3bbd0ccb46a4490ab903198e56594c1f (127.17.130.1:37903): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6affaf75288f4bb8a2f56331085420f2: Network error: Client connection negotiation failed: client connection to 127.17.130.1:37903: connect: Connection refused (error 111)
I20260812 01:13:09.207163 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 17875
I20260812 01:13:09.223789 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 17247
2026-08-12T01:13:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2548 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T01:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:09Z Disabled control of system clock
I20260812 01:13:09.273460 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46117
--webserver_interface=127.17.130.62
--webserver_port=0
--builtin_ntp_servers=127.17.130.20:35437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:46117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:09.372726 18202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:09.373003 18202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:09.373044 18202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:09.374526 18202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:09.374585 18202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:09.374599 18202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:09.374612 18202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:09.374625 18202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:09.376108 18202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:46117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:46117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:09.376341 18202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:09.376756 18202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:09.382706 18210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.382707 18213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.382707 18209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:09.382740 18202 server_base.cc:1034] running on GCE node
I20260812 01:13:09.383402 18202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:09.383723 18202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:09.384922 18202 hybrid_clock.cc:648] HybridClock initialized: now 1786497189384893 us; error 53 us; skew 500 ppm
I20260812 01:13:09.386381 18202 webserver.cc:460] Webserver started at http://127.17.130.62:37937/ using document root <none> and password file <none>
I20260812 01:13:09.386689 18202 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:09.386772 18202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:09.386962 18202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:09.388132 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "78b756e9651342d8b52810cdf43f3dca"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "4fc7ae7e4ff6bef75bc057d99cf98824"
server_key_iv: "024643c5e37c9a1fb06c52438203f9ef"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.388532 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "78b756e9651342d8b52810cdf43f3dca"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "4fc7ae7e4ff6bef75bc057d99cf98824"
server_key_iv: "024643c5e37c9a1fb06c52438203f9ef"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.390419 18202 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260812 01:13:09.391367 18226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.391636 18202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:09.391764 18202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "78b756e9651342d8b52810cdf43f3dca"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "4fc7ae7e4ff6bef75bc057d99cf98824"
server_key_iv: "024643c5e37c9a1fb06c52438203f9ef"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.391850 18202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:09.404194 18202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:09.404570 18202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:09.404742 18202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:09.409207 18202 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:46117
I20260812 01:13:09.409279 18326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:46117 every 8 connection(s)
I20260812 01:13:09.409722 18202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:09.410987 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:09.413632 18327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca: Bootstrap starting.
I20260812 01:13:09.414304 18327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:09.414649 18327 log.cc:824] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:09.415488 18327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca: No bootstrap required, opened a new log
I20260812 01:13:09.417467 18327 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b756e9651342d8b52810cdf43f3dca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46117 } }
I20260812 01:13:09.417627 18327 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:09.417660 18327 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78b756e9651342d8b52810cdf43f3dca, State: Initialized, Role: FOLLOWER
I20260812 01:13:09.417862 18327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b756e9651342d8b52810cdf43f3dca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46117 } }
I20260812 01:13:09.417960 18327 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:09.417987 18327 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:09.418016 18327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:09.419275 18327 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b756e9651342d8b52810cdf43f3dca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46117 } }
I20260812 01:13:09.419386 18327 leader_election.cc:302] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78b756e9651342d8b52810cdf43f3dca; no voters:
I20260812 01:13:09.419662 18327 leader_election.cc:288] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:09.419806 18330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:09.420001 18327 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:09.420053 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18202
I20260812 01:13:09.420253 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:09.420495 17928 external_mini_cluster.cc:1428] Setting key 65ed845465dc94dd71ea7df3b6d3a20e
I20260812 01:13:09.420686 18330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [term 1 LEADER]: Becoming Leader. State: Replica: 78b756e9651342d8b52810cdf43f3dca, State: Running, Role: LEADER
I20260812 01:13:09.420825 18330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b756e9651342d8b52810cdf43f3dca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46117 } }
I20260812 01:13:09.422469 18332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78b756e9651342d8b52810cdf43f3dca. Latest consensus state: current_term: 1 leader_uuid: "78b756e9651342d8b52810cdf43f3dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b756e9651342d8b52810cdf43f3dca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46117 } } }
I20260812 01:13:09.422334 18331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78b756e9651342d8b52810cdf43f3dca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78b756e9651342d8b52810cdf43f3dca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 46117 } } }
I20260812 01:13:09.422577 18332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:09.422610 18331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:09.423125 18340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:09.425199 18340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:09.427188 18340 catalog_manager.cc:1347] Generated new cluster ID: 16f0dbd7734b4e0ebeaffa66bc8fe177
I20260812 01:13:09.427239 18340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:09.430631 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.422583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.17.130.62:46117)
0812 01:13:09.423226 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.423237 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.423302 (+ 65us) server_negotiation.cc:407] Connection header received
0812 01:13:09.423450 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.423457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.423540 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.424503 (+ 963us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:09.424610 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.426436 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.427766 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.428465 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.429093 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.429490 (+ 397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:09.429494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:09.429494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:09.429744 (+ 250us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:09.429787 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:09.429803 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:09.430112 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.430154 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.430275 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":128,"threads_started":1}
I20260812 01:13:09.450630 18340 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:09.451251 18340 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:09.457023 18340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca: Generated new TSK 0
I20260812 01:13:09.457265 18340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:09.464003 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.1:0
--local_ip_for_outbound_sockets=127.17.130.1
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46117
--builtin_ntp_servers=127.17.130.20:35437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 01:13:09.595151 18352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:09.595484 18352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:09.595522 18352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:09.597530 18352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:09.597610 18352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:09.597654 18352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.1
I20260812 01:13:09.599987 18352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.17.130.1
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.1
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:09.600307 18352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:09.600723 18352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:09.607227 18361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.607193 18359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.607319 18364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:09.607762 18352 server_base.cc:1034] running on GCE node
I20260812 01:13:09.608057 18352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:09.608326 18352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:09.609524 18352 hybrid_clock.cc:648] HybridClock initialized: now 1786497189609501 us; error 43 us; skew 500 ppm
I20260812 01:13:09.611157 18352 webserver.cc:460] Webserver started at http://127.17.130.1:33253/ using document root <none> and password file <none>
I20260812 01:13:09.611433 18352 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:09.611523 18352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:09.611697 18352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:09.612861 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data/instance:
uuid: "9538313c0534448e9d0d862684e16918"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "11f8fb30e982432ef0b1e9e7ed47b789"
server_key_iv: "45fb8d6268c0a19311091ca6bf796377"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.613294 18352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance:
uuid: "9538313c0534448e9d0d862684e16918"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "11f8fb30e982432ef0b1e9e7ed47b789"
server_key_iv: "45fb8d6268c0a19311091ca6bf796377"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.614943 18352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:09.616062 18371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.616266 18352 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:09.616369 18352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal
uuid: "9538313c0534448e9d0d862684e16918"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "11f8fb30e982432ef0b1e9e7ed47b789"
server_key_iv: "45fb8d6268c0a19311091ca6bf796377"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.616467 18352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:09.650502 18352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:09.650884 18352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:09.651010 18352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:09.651321 18352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:09.651713 18352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:09.651749 18352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.651772 18352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:09.651815 18352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.658870 18352 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.1:34485
I20260812 01:13:09.658913 18560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.1:34485 every 8 connection(s)
I20260812 01:13:09.659337 18352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/data/info.pb
I20260812 01:13:09.662600 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18352
I20260812 01:13:09.662701 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-0/wal/instance
I20260812 01:13:09.662907 17928 external_mini_cluster.cc:1428] Setting key 3bd2d11ac3a86904da9bc3cdc76d9da3
I20260812 01:13:09.664430 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.2:0
--local_ip_for_outbound_sockets=127.17.130.2
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46117
--builtin_ntp_servers=127.17.130.20:35437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:13:09.666563 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.660275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.1:59011 (local address 127.17.130.62:46117)
0812 01:13:09.660338 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.660364 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.660600 (+ 236us) server_negotiation.cc:407] Connection header received
0812 01:13:09.661237 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.661238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.661254 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.661319 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:09.662570 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.663428 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.664560 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.664862 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.664985 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.666007 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:09.666009 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:09.666009 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:09.666015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:09.666031 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:09.666153 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:09.666240 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.666409 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.666450 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 01:13:09.666709 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.660231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46117 (local address 127.17.130.1:59011)
0812 01:13:09.660472 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.660510 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.661117 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.661445 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.661598 (+ 153us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.661610 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:09.662419 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.662427 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.663519 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.663520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.664457 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.664460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.665005 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.665006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.665215 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.665843 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:09.665844 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:09.665876 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:09.665901 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:09.666269 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:09.666279 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:09.666284 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.666379 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.666487 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260812 01:13:09.667248 18562 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46117
I20260812 01:13:09.667371 18562 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:09.667598 18562 heartbeater.cc:511] Master 127.17.130.62:46117 requested a full tablet report, sending...
I20260812 01:13:09.668231 18256 ts_manager.cc:194] Registered new tserver with Master: 9538313c0534448e9d0d862684e16918 (127.17.130.1:34485)
I20260812 01:13:09.668876 18256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.1:59011
W20260812 01:13:09.758100 18567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:09.758396 18567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:09.758442 18567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:09.759680 18567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:09.759758 18567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:09.759804 18567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.2
I20260812 01:13:09.761329 18567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.17.130.2
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.2
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:09.761631 18567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:09.761976 18567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:09.767635 18573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.767701 18575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.767714 18577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:09.767832 18567 server_base.cc:1034] running on GCE node
I20260812 01:13:09.768141 18567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:09.768415 18567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:09.769753 18567 hybrid_clock.cc:648] HybridClock initialized: now 1786497189769729 us; error 39 us; skew 500 ppm
I20260812 01:13:09.771353 18567 webserver.cc:460] Webserver started at http://127.17.130.2:42855/ using document root <none> and password file <none>
I20260812 01:13:09.771639 18567 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:09.771718 18567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:09.771878 18567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:09.773156 18567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data/instance:
uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "0dea298334c5c6b603c1c9872e90f219"
server_key_iv: "8679ee640003ae37189118592907782a"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.773568 18567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance:
uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "0dea298334c5c6b603c1c9872e90f219"
server_key_iv: "8679ee640003ae37189118592907782a"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.775180 18567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:13:09.776090 18588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.776340 18567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:09.776521 18567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal
uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "0dea298334c5c6b603c1c9872e90f219"
server_key_iv: "8679ee640003ae37189118592907782a"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.776660 18567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:09.799481 18567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:09.799862 18567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:09.800040 18567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:09.800305 18567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:09.800698 18567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:09.800757 18567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.800793 18567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:09.800836 18567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.807011 18567 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.2:41015
I20260812 01:13:09.807092 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.2:41015 every 8 connection(s)
I20260812 01:13:09.807441 18567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/data/info.pb
I20260812 01:13:09.812453 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18567
I20260812 01:13:09.812587 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal/instance
I20260812 01:13:09.812795 17928 external_mini_cluster.cc:1428] Setting key 27c003a91eefec9c29ebe3ad04bad833
I20260812 01:13:09.813380 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.808277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:50423 (local address 127.17.130.62:46117)
0812 01:13:09.808348 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.808350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.808592 (+ 242us) server_negotiation.cc:407] Connection header received
0812 01:13:09.809186 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.809188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.809207 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.809289 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:09.810096 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.810610 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.811578 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.811764 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.811825 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.812919 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:09.812921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:09.812921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:09.812927 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:09.812946 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:09.812951 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:09.813023 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.813266 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.813298 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:13:09.813602 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.808193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46117 (local address 127.17.130.2:50423)
0812 01:13:09.808389 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.808435 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.808946 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.809405 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.809409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.809418 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:09.809975 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.809980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.810708 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.810708 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.811484 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.811486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.811875 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.811876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.812035 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.812740 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:09.812740 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:09.812773 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:09.812796 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:09.813151 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:09.813157 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:09.813160 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.813241 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.813351 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260812 01:13:09.813913 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.17.130.3:0
--local_ip_for_outbound_sockets=127.17.130.3
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46117
--builtin_ntp_servers=127.17.130.20:35437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 01:13:09.814152 18761 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46117
I20260812 01:13:09.814248 18761 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:09.814452 18761 heartbeater.cc:511] Master 127.17.130.62:46117 requested a full tablet report, sending...
I20260812 01:13:09.814870 18256 ts_manager.cc:194] Registered new tserver with Master: 60cc3e4ea43a4a0e8a95e6a65d254cd9 (127.17.130.2:41015)
I20260812 01:13:09.815402 18256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.2:50423
W20260812 01:13:09.914601 18764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:09.914853 18764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:09.914878 18764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:09.916090 18764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:09.916136 18764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:09.916162 18764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.17.130.3
I20260812 01:13:09.917847 18764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.17.130.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.17.130.3
--webserver_port=0
--tserver_master_addrs=127.17.130.62:46117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.17.130.3
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:09.918279 18764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:09.918669 18764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:09.925287 18777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.925309 18772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:09.925319 18773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:09.925562 18764 server_base.cc:1034] running on GCE node
I20260812 01:13:09.925818 18764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:09.926167 18764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:09.927393 18764 hybrid_clock.cc:648] HybridClock initialized: now 1786497189927371 us; error 30 us; skew 500 ppm
I20260812 01:13:09.933411 18764 webserver.cc:460] Webserver started at http://127.17.130.3:41029/ using document root <none> and password file <none>
I20260812 01:13:09.933737 18764 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:09.933795 18764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:09.933950 18764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:09.935209 18764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data/instance:
uuid: "e06306a3a9a04880817bb2d9f6b23266"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "eeb72e4bdd74ba2da9b7fda1852427b1"
server_key_iv: "cbf7b97e36b170eb7c4dc81e113cda79"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.935608 18764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance:
uuid: "e06306a3a9a04880817bb2d9f6b23266"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "eeb72e4bdd74ba2da9b7fda1852427b1"
server_key_iv: "cbf7b97e36b170eb7c4dc81e113cda79"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.937232 18764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260812 01:13:09.938349 18784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.938640 18764 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:09.938769 18764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal
uuid: "e06306a3a9a04880817bb2d9f6b23266"
format_stamp: "Formatted at 2026-08-12 01:13:09 on dist-test-slave-tfpf"
server_key: "eeb72e4bdd74ba2da9b7fda1852427b1"
server_key_iv: "cbf7b97e36b170eb7c4dc81e113cda79"
server_key_version: "encryptionkey@0"
I20260812 01:13:09.938865 18764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:09.966048 18764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:09.966419 18764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:09.966591 18764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:09.966861 18764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 01:13:09.967235 18764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 01:13:09.967293 18764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.967330 18764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 01:13:09.967377 18764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:09.973834 18764 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.3:35253
I20260812 01:13:09.973888 18994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.3:35253 every 8 connection(s)
I20260812 01:13:09.974288 18764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/data/info.pb
I20260812 01:13:09.980777 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.975594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.3:59255 (local address 127.17.130.62:46117)
0812 01:13:09.975649 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.975651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.976042 (+ 391us) server_negotiation.cc:407] Connection header received
0812 01:13:09.976547 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.976548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.976560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.976620 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:09.977602 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.978117 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.979218 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.979418 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.979486 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.980407 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:09.980408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:09.980409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:09.980414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:09.980428 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:09.980432 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:09.980493 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.980680 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.980710 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 01:13:09.980952 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.975528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:46117 (local address 127.17.130.3:59255)
0812 01:13:09.975839 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.975887 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.976417 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.976779 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.976783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.976801 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:09.977487 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.977492 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.978212 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.978213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.979098 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.979101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.979496 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.979497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.979607 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.980237 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 01:13:09.980237 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:09.980272 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 01:13:09.980297 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:09.980543 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:09.980549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:09.980552 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.980624 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.980713 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":107,"threads_started":1}
I20260812 01:13:09.981671 18995 heartbeater.cc:347] Connected to a master server at 127.17.130.62:46117
I20260812 01:13:09.981791 18995 heartbeater.cc:464] Registering TS with master...
I20260812 01:13:09.981993 18995 heartbeater.cc:511] Master 127.17.130.62:46117 requested a full tablet report, sending...
I20260812 01:13:09.982134 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 18764
I20260812 01:13:09.982267 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal/instance
I20260812 01:13:09.982458 18256 ts_manager.cc:194] Registered new tserver with Master: e06306a3a9a04880817bb2d9f6b23266 (127.17.130.3:35253)
I20260812 01:13:09.982499 17928 external_mini_cluster.cc:1428] Setting key c49d0461f75e9007839dd78baf0e0d9b
I20260812 01:13:09.982972 18256 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.17.130.3:59255
I20260812 01:13:09.983403 17928 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 01:13:09.988309 18341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.984835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.17.130.62:46117)
0812 01:13:09.984882 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.984885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.985138 (+ 253us) server_negotiation.cc:407] Connection header received
0812 01:13:09.985201 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.985202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.985214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.985274 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:09.985391 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.985843 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.986619 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.986809 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.986882 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.987884 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:09.987885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:09.987885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:09.987890 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:09.987905 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:09.987919 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:09.987982 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.988139 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.988196 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:09.990319 18256 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 01:13:09.991925 18256 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 01:13:09.999414 19011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.994608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34485 (local address 127.0.0.1:59592)
0812 01:13:09.994741 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.994782 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.994865 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.995631 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.995632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.995634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:09.995697 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.995699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.997112 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.997113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.998346 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.998348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.999046 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.999047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.999160 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.999176 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.999229 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.999291 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":29,"thread_start_us":58,"threads_started":1}
I20260812 01:13:09.999424 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.995296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.17.130.2:41015)
0812 01:13:09.995640 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.995644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.995653 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:09.995692 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.995694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.995709 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.995790 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:09.995912 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.996994 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.998138 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.998883 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.998960 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.999076 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.999302 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.999343 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":115,"threads_started":1}
I20260812 01:13:09.999462 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.994941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.17.130.3:35253)
0812 01:13:09.995311 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.995315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.995326 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:13:09.995365 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.995367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.995384 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.995456 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:09.995702 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.996601 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.998233 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.998935 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.999001 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.999075 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.999341 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.999382 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":99,"threads_started":1}
I20260812 01:13:09.999676 19013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.994998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.2:41015 (local address 127.0.0.1:41152)
0812 01:13:09.995360 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.995372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.995430 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.995801 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.995802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.995804 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:09.995846 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.995847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.997134 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.997140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.998051 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.998054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.999116 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.999117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.999173 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.999177 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.999280 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.999325 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260812 01:13:09.999758 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.994828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:35253 (local address 127.0.0.1:56752)
0812 01:13:09.995169 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:09.995182 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:09.995238 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:09.995543 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:09.995546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:09.995548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:09.995621 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.995624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.996693 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.996694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.998133 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:09.998137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.999045 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.999046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:09.999170 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.999177 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 01:13:09.999249 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 01:13:09.999290 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":42,"thread_start_us":79,"threads_started":1}
I20260812 01:13:10.000034 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:09.994765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59592 (local address 127.17.130.1:34485)
0812 01:13:09.995177 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:09.995188 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:09.995198 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:09.995482 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:09.995489 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:09.995512 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:09.995777 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:09.995845 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.996987 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.998425 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:09.998934 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:09.999378 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:09.999864 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:09.999901 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 01:13:09.999954 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":123,"threads_started":1}
I20260812 01:13:10.000559 18866 tablet_service.cc:1467] Processing CreateTablet for tablet 9577847cd38c45fa803cbf7548082bb3 (DEFAULT_TABLE table=test-table [id=b33657c4a43547cb86f171a5c5aa4d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:10.000808 18458 tablet_service.cc:1467] Processing CreateTablet for tablet 9577847cd38c45fa803cbf7548082bb3 (DEFAULT_TABLE table=test-table [id=b33657c4a43547cb86f171a5c5aa4d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:10.001000 18866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9577847cd38c45fa803cbf7548082bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:10.000811 18653 tablet_service.cc:1467] Processing CreateTablet for tablet 9577847cd38c45fa803cbf7548082bb3 (DEFAULT_TABLE table=test-table [id=b33657c4a43547cb86f171a5c5aa4d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 01:13:10.001159 18458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9577847cd38c45fa803cbf7548082bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:10.001344 18653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9577847cd38c45fa803cbf7548082bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:10.005118 19019 tablet_bootstrap.cc:492] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Bootstrap starting.
I20260812 01:13:10.005745 19022 tablet_bootstrap.cc:492] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: Bootstrap starting.
I20260812 01:13:10.006054 19019 tablet_bootstrap.cc:654] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:10.006325 19019 log.cc:824] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:10.006439 19022 tablet_bootstrap.cc:654] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:10.006757 19022 log.cc:824] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:10.007179 19023 tablet_bootstrap.cc:492] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Bootstrap starting.
I20260812 01:13:10.007354 19019 tablet_bootstrap.cc:492] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: No bootstrap required, opened a new log
I20260812 01:13:10.007436 19019 ts_tablet_manager.cc:1397] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:10.007827 19023 tablet_bootstrap.cc:654] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:10.008139 19023 log.cc:824] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:10.008749 19022 tablet_bootstrap.cc:492] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: No bootstrap required, opened a new log
I20260812 01:13:10.008831 19022 ts_tablet_manager.cc:1397] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 01:13:10.008963 19023 tablet_bootstrap.cc:492] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: No bootstrap required, opened a new log
I20260812 01:13:10.009088 19023 ts_tablet_manager.cc:1397] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 01:13:10.009361 19019 raft_consensus.cc:348] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.009503 19019 raft_consensus.cc:374] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:10.009526 19019 raft_consensus.cc:729] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60cc3e4ea43a4a0e8a95e6a65d254cd9, State: Initialized, Role: FOLLOWER
I20260812 01:13:10.009653 19019 consensus_queue.cc:260] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.009982 18761 heartbeater.cc:503] Master 127.17.130.62:46117 was elected leader, sending a full tablet report...
I20260812 01:13:10.010238 19019 ts_tablet_manager.cc:1428] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 01:13:10.011642 19022 raft_consensus.cc:348] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.011835 19022 raft_consensus.cc:374] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:10.011879 19022 raft_consensus.cc:729] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9538313c0534448e9d0d862684e16918, State: Initialized, Role: FOLLOWER
I20260812 01:13:10.012053 19022 consensus_queue.cc:260] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.012146 19023 raft_consensus.cc:348] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.012310 19023 raft_consensus.cc:374] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:10.012355 19022 ts_tablet_manager.cc:1428] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260812 01:13:10.012352 19023 raft_consensus.cc:729] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e06306a3a9a04880817bb2d9f6b23266, State: Initialized, Role: FOLLOWER
I20260812 01:13:10.012414 18562 heartbeater.cc:503] Master 127.17.130.62:46117 was elected leader, sending a full tablet report...
I20260812 01:13:10.012471 19023 consensus_queue.cc:260] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.012764 19023 ts_tablet_manager.cc:1428] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260812 01:13:10.012794 18995 heartbeater.cc:503] Master 127.17.130.62:46117 was elected leader, sending a full tablet report...
W20260812 01:13:10.058284 18762 tablet.cc:2369] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:10.135504 19030 raft_consensus.cc:482] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 01:13:10.135637 19030 raft_consensus.cc:504] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.136063 19030 leader_election.cc:288] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9538313c0534448e9d0d862684e16918 (127.17.130.1:34485), e06306a3a9a04880817bb2d9f6b23266 (127.17.130.3:35253)
I20260812 01:13:10.139675 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:10.136539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:42963 (local address 127.17.130.3:35253)
0812 01:13:10.136611 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:10.136614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:10.136725 (+ 111us) server_negotiation.cc:407] Connection header received
0812 01:13:10.136841 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:10.136842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:10.136851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:10.136922 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:10.137107 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.137791 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.138844 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.139239 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.139330 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:10.139340 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:10.139550 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 01:13:10.139595 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 01:13:10.139724 19033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:10.136418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.3:35253 (local address 127.17.130.2:42963)
0812 01:13:10.136650 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:10.136660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:10.136753 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:10.136937 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:10.136937 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:10.136939 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:10.137012 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:10.137013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.137888 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.137889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:10.138735 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:10.138738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.139368 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.139369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:10.139465 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:10.139472 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 01:13:10.139530 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 01:13:10.139588 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":108,"threads_started":1}
I20260812 01:13:10.140141 18897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9577847cd38c45fa803cbf7548082bb3" candidate_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06306a3a9a04880817bb2d9f6b23266" is_pre_election: true
I20260812 01:13:10.140327 18897 raft_consensus.cc:2454] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60cc3e4ea43a4a0e8a95e6a65d254cd9 in term 0.
I20260812 01:13:10.140355 19012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:10.136379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.17.130.2:58001 (local address 127.17.130.1:34485)
0812 01:13:10.136431 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:10.136432 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:10.136568 (+ 136us) server_negotiation.cc:407] Connection header received
0812 01:13:10.136609 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:10.136610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:10.136623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:10.136697 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 01:13:10.136912 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.137697 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.139323 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.139861 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.139944 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:10.139954 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:10.140236 (+ 282us) server_negotiation.cc:299] Negotiation successful
0812 01:13:10.140285 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 01:13:10.140369 18763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:10.136307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34485 (local address 127.17.130.2:58001)
0812 01:13:10.136397 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:10.136411 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:10.136496 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:10.136728 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:10.136730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:10.136733 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 01:13:10.136819 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:10.136821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.137796 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.137798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:10.139206 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:10.139210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.140063 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.140064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:10.140156 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:10.140161 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 01:13:10.140216 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 01:13:10.140271 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 01:13:10.140601 18591 leader_election.cc:302] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60cc3e4ea43a4a0e8a95e6a65d254cd9, e06306a3a9a04880817bb2d9f6b23266; no voters:
I20260812 01:13:10.140829 19030 raft_consensus.cc:2781] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 01:13:10.140918 19030 raft_consensus.cc:482] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 01:13:10.140952 19030 raft_consensus.cc:3037] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:10.140913 18487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9577847cd38c45fa803cbf7548082bb3" candidate_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9538313c0534448e9d0d862684e16918" is_pre_election: true
I20260812 01:13:10.141127 18487 raft_consensus.cc:2454] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60cc3e4ea43a4a0e8a95e6a65d254cd9 in term 0.
I20260812 01:13:10.142212 19030 raft_consensus.cc:504] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.142376 19030 leader_election.cc:288] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [CANDIDATE]: Term 1 election: Requested vote from peers 9538313c0534448e9d0d862684e16918 (127.17.130.1:34485), e06306a3a9a04880817bb2d9f6b23266 (127.17.130.3:35253)
I20260812 01:13:10.142537 18487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9577847cd38c45fa803cbf7548082bb3" candidate_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9538313c0534448e9d0d862684e16918"
I20260812 01:13:10.142573 18897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9577847cd38c45fa803cbf7548082bb3" candidate_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e06306a3a9a04880817bb2d9f6b23266"
I20260812 01:13:10.142655 18487 raft_consensus.cc:3037] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:10.142680 18897 raft_consensus.cc:3037] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:10.143630 18487 raft_consensus.cc:2454] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60cc3e4ea43a4a0e8a95e6a65d254cd9 in term 1.
I20260812 01:13:10.143821 18591 leader_election.cc:302] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60cc3e4ea43a4a0e8a95e6a65d254cd9, 9538313c0534448e9d0d862684e16918; no voters:
I20260812 01:13:10.143997 19030 raft_consensus.cc:2781] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:10.144714 18897 raft_consensus.cc:2454] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60cc3e4ea43a4a0e8a95e6a65d254cd9 in term 1.
I20260812 01:13:10.144747 19030 raft_consensus.cc:686] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 1 LEADER]: Becoming Leader. State: Replica: 60cc3e4ea43a4a0e8a95e6a65d254cd9, State: Running, Role: LEADER
I20260812 01:13:10.144910 19030 consensus_queue.cc:237] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } }
I20260812 01:13:10.146162 18256 catalog_manager.cc:5495] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60cc3e4ea43a4a0e8a95e6a65d254cd9 (127.17.130.2). New cstate: current_term: 1 leader_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60cc3e4ea43a4a0e8a95e6a65d254cd9" member_type: VOTER last_known_addr { host: "127.17.130.2" port: 41015 } health_report { overall_health: HEALTHY } } }
W20260812 01:13:10.160216 18563 tablet.cc:2369] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 01:13:10.183516 19017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:10.181002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41162 (local address 127.17.130.2:41015)
0812 01:13:10.181128 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:10.181130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:10.181139 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:10.181178 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:10.181179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:10.181204 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:10.181292 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:10.181413 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.181937 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.182720 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:10.182920 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:10.183005 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:10.183155 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:10.183270 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:10.183366 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:10.183388 (+ 22us) server_negotiation.cc:299] Negotiation successful
0812 01:13:10.183411 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 01:13:10.186496 18487 raft_consensus.cc:1261] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 1 FOLLOWER]: Refusing update from remote peer 60cc3e4ea43a4a0e8a95e6a65d254cd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:10.186496 18897 raft_consensus.cc:1261] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 1 FOLLOWER]: Refusing update from remote peer 60cc3e4ea43a4a0e8a95e6a65d254cd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:10.186962 19030 consensus_queue.cc:1035] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9538313c0534448e9d0d862684e16918" member_type: VOTER last_known_addr { host: "127.17.130.1" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:10.187202 19035 consensus_queue.cc:1035] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e06306a3a9a04880817bb2d9f6b23266" member_type: VOTER last_known_addr { host: "127.17.130.3" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:10.190083 19044 mvcc.cc:205] Tried to move back new op lower bound from 7317492490998378496 to 7317492490834821120. Current Snapshot: MvccSnapshot[applied={T|T < 7317492490998378496}]
I20260812 01:13:10.190519 19041 mvcc.cc:205] Tried to move back new op lower bound from 7317492490998378496 to 7317492490834821120. Current Snapshot: MvccSnapshot[applied={T|T < 7317492490998378496}]
I20260812 01:13:10.192489 18252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:10.192620 18252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 01:13:10.193512 18252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 78b756e9651342d8b52810cdf43f3dca: Sending DeleteTablet for 3 replicas of tablet 9577847cd38c45fa803cbf7548082bb3
I20260812 01:13:10.193913 18458 tablet_service.cc:1514] Processing DeleteTablet for tablet 9577847cd38c45fa803cbf7548082bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:10 UTC) from {username='slave'} at 127.0.0.1:59592
I20260812 01:13:10.193946 18653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9577847cd38c45fa803cbf7548082bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:10 UTC) from {username='slave'} at 127.0.0.1:41152
I20260812 01:13:10.193948 18866 tablet_service.cc:1514] Processing DeleteTablet for tablet 9577847cd38c45fa803cbf7548082bb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 01:13:10 UTC) from {username='slave'} at 127.0.0.1:56752
I20260812 01:13:10.194344 19054 tablet_replica.cc:330] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918: stopping tablet replica
I20260812 01:13:10.194527 19054 raft_consensus.cc:2229] T 9577847cd38c45fa803cbf7548082bb3 P 9538313c0534448e9d0d862684e16918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:10.194571 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18352
I20260812 01:13:10.194324 19055 tablet_replica.cc:330] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: stopping tablet replica
I20260812 01:13:10.211783 19055 raft_consensus.cc:2229] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 01:13:10.211935 19055 raft_consensus.cc:2258] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:10.212163 19056 tablet_replica.cc:330] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: stopping tablet replica
I20260812 01:13:10.212296 19056 raft_consensus.cc:2229] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 01:13:10.212415 19056 raft_consensus.cc:2258] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 01:13:10.214140 19056 ts_tablet_manager.cc:1905] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:10.214283 19055 ts_tablet_manager.cc:1905] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 01:13:10.216545 19055 ts_tablet_manager.cc:1918] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:10.216599 19056 ts_tablet_manager.cc:1918] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 01:13:10.216722 19056 log.cc:1196] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-2/wal/wals/9577847cd38c45fa803cbf7548082bb3
I20260812 01:13:10.216842 19055 log.cc:1196] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Deleting WAL directory at /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786496918848844-17928-0/minicluster-data/ts-1/wal/wals/9577847cd38c45fa803cbf7548082bb3
I20260812 01:13:10.217025 19056 ts_tablet_manager.cc:1939] T 9577847cd38c45fa803cbf7548082bb3 P e06306a3a9a04880817bb2d9f6b23266: Deleting consensus metadata
I20260812 01:13:10.217322 19055 ts_tablet_manager.cc:1939] T 9577847cd38c45fa803cbf7548082bb3 P 60cc3e4ea43a4a0e8a95e6a65d254cd9: Deleting consensus metadata
I20260812 01:13:10.217676 18231 catalog_manager.cc:4841] TS e06306a3a9a04880817bb2d9f6b23266 (127.17.130.3:35253): tablet 9577847cd38c45fa803cbf7548082bb3 (table test-table [id=b33657c4a43547cb86f171a5c5aa4d62]) successfully deleted
I20260812 01:13:10.217963 18231 catalog_manager.cc:4841] TS 60cc3e4ea43a4a0e8a95e6a65d254cd9 (127.17.130.2:41015): tablet 9577847cd38c45fa803cbf7548082bb3 (table test-table [id=b33657c4a43547cb86f171a5c5aa4d62]) successfully deleted
W20260812 01:13:10.219944 18231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.17.130.1:34485 (error 108)
I20260812 01:13:10.220621 19015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:10.220452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.1:34485 (local address 127.0.0.1:59606)
0812 01:13:10.220528 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:10.220577 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 01:13:10.220690 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18567
W20260812 01:13:10.227828 18231 catalog_manager.cc:4584] TS 9538313c0534448e9d0d862684e16918 (127.17.130.1:34485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9577847cd38c45fa803cbf7548082bb3: Network error: Client connection negotiation failed: client connection to 127.17.130.1:34485: connect: Connection refused (error 111)
I20260812 01:13:10.233922 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18764
I20260812 01:13:10.246433 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 18202
2026-08-12T01:13:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1019 ms)
[----------] 6 tests from EncryptionPolicyTest (10875 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:10 dist-test-slave-tfpf krb5kdc[19062](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:10 dist-test-slave-tfpf krb5kdc[19062](info): set up 2 sockets
Aug 12 01:13:10 dist-test-slave-tfpf krb5kdc[19062](info): commencing operation
krb5kdc: starting...
W20260812 01:13:11.622238 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:11 dist-test-slave-tfpf krb5kdc[19062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:11.764345 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34057
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:11.855078 19085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:11.855305 19085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:11.855330 19085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:11.856408 19085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:11.856451 19085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:11.856465 19085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:11.856477 19085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:11.856489 19085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:11.858311 19085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34057
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:11.858577 19085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:11.858771 19085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:11.864413 19092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:11.864344 19093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:11.864583 19085 server_base.cc:1034] running on GCE node
W20260812 01:13:11.864357 19096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:11.865262 19085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:11.865561 19085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:11.866752 19085 hybrid_clock.cc:648] HybridClock initialized: now 1786497191866737 us; error 32 us; skew 500 ppm
Aug 12 01:13:11 dist-test-slave-tfpf krb5kdc[19062](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:11.869583 19085 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:11.870136 19085 webserver.cc:460] Webserver started at http://127.0.0.1:40367/ using document root <none> and password file <none>
I20260812 01:13:11.870378 19085 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:11.870448 19085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:11.870618 19085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:11.871778 19085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "12e717dce31e499ebdabe58fb3ad7b68"
format_stamp: "Formatted at 2026-08-12 01:13:11 on dist-test-slave-tfpf"
server_key: "9b09b547ebf1891d73728817d3965271"
server_key_iv: "aff3adecf567fef1228fd40b2c88e2a0"
server_key_version: "encryptionkey@0"
I20260812 01:13:11.872179 19085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "12e717dce31e499ebdabe58fb3ad7b68"
format_stamp: "Formatted at 2026-08-12 01:13:11 on dist-test-slave-tfpf"
server_key: "9b09b547ebf1891d73728817d3965271"
server_key_iv: "aff3adecf567fef1228fd40b2c88e2a0"
server_key_version: "encryptionkey@0"
I20260812 01:13:11.873813 19085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:11.874766 19110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:11.875033 19085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:11.875164 19085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "12e717dce31e499ebdabe58fb3ad7b68"
format_stamp: "Formatted at 2026-08-12 01:13:11 on dist-test-slave-tfpf"
server_key: "9b09b547ebf1891d73728817d3965271"
server_key_iv: "aff3adecf567fef1228fd40b2c88e2a0"
server_key_version: "encryptionkey@0"
I20260812 01:13:11.875250 19085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:11.888612 19085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:11.891254 19085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:11.891422 19085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:11.897950 19085 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34057
I20260812 01:13:11.898142 19205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34057 every 8 connection(s)
I20260812 01:13:11.898439 19085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:11.899503 19206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:11.901121 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19085
I20260812 01:13:11.901250 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:11.901443 17928 external_mini_cluster.cc:1428] Setting key b1239f6dc1dba3375958a23df9bc785b
I20260812 01:13:11.903492 19206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68: Bootstrap starting.
I20260812 01:13:11.904497 19206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:11.904915 19206 log.cc:824] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:11.906129 19206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68: No bootstrap required, opened a new log
I20260812 01:13:11.909444 19206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e717dce31e499ebdabe58fb3ad7b68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34057 } }
I20260812 01:13:11.909621 19206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:11.909644 19206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e717dce31e499ebdabe58fb3ad7b68, State: Initialized, Role: FOLLOWER
Aug 12 01:13:11 dist-test-slave-tfpf krb5kdc[19062](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:13:11.909817 19206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e717dce31e499ebdabe58fb3ad7b68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34057 } }
I20260812 01:13:11.909907 19206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:11.909958 19206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:11.910012 19206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:11.910717 19206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e717dce31e499ebdabe58fb3ad7b68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34057 } }
I20260812 01:13:11.910838 19206 leader_election.cc:302] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e717dce31e499ebdabe58fb3ad7b68; no voters:
I20260812 01:13:11.911053 19206 leader_election.cc:288] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:11.911082 19213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:11.911216 19213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [term 1 LEADER]: Becoming Leader. State: Replica: 12e717dce31e499ebdabe58fb3ad7b68, State: Running, Role: LEADER
I20260812 01:13:11.911310 19213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e717dce31e499ebdabe58fb3ad7b68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34057 } }
I20260812 01:13:11.911381 19206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:11.911643 19213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e717dce31e499ebdabe58fb3ad7b68. Latest consensus state: current_term: 1 leader_uuid: "12e717dce31e499ebdabe58fb3ad7b68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e717dce31e499ebdabe58fb3ad7b68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34057 } } }
I20260812 01:13:11.911684 19215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e717dce31e499ebdabe58fb3ad7b68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e717dce31e499ebdabe58fb3ad7b68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34057 } } }
I20260812 01:13:11.911736 19213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:11.911791 19215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:11.911976 19221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:11.912547 19221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:11.914489 19221 catalog_manager.cc:1347] Generated new cluster ID: 1d6de0a528fa4f36b2e0668003ce7f0f
I20260812 01:13:11.914546 19221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:11.915114 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:11.902176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43521 (local address 127.0.0.1:34057)
0812 01:13:11.902455 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:11.902465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:11.902499 (+ 34us) server_negotiation.cc:407] Connection header received
0812 01:13:11.902981 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:11.902994 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:11.903053 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:11.903226 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:11.903946 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:11.905002 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:11.906298 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:11.906778 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:11.906883 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:11.910389 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:11.910393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:11.910393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:11.910407 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:11.912847 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:11.913413 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:11.913417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:11.913417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:11.913572 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:11.913899 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:11.913900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:11.913901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:11.914245 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:11.914426 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:11.914797 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 01:13:11.914908 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":145,"threads_started":1}
I20260812 01:13:11.916132 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:11.925467 19210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:11.917762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39929 (local address 127.0.0.1:34057)
0812 01:13:11.917802 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:11.917803 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:11.917851 (+ 48us) server_negotiation.cc:407] Connection header received
0812 01:13:11.917914 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:11.917915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:11.917925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:11.917996 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:11.918622 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:11.919503 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:11.920484 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:11.920835 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:11.920908 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:11.922424 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:11.922425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:11.922426 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:11.922433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:11.924096 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:11.924556 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:11.924557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:11.924558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:11.924625 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:11.924866 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:11.924867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:11.924867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:11.925006 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:11.925142 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:11.925291 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 01:13:11.925360 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:13:11.938063 19221 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:11.938835 19221 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:11.954879 19221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e717dce31e499ebdabe58fb3ad7b68: Generated new TSK 0
I20260812 01:13:11.955104 19221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:11.959846 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19085
W20260812 01:13:11.973109 19059 connection.cc:535] client connection to 127.0.0.1:34057 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34057): Connection reset by peer (error 104)
2026-08-12T01:13:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1704 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:12 dist-test-slave-tfpf krb5kdc[19259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:12 dist-test-slave-tfpf krb5kdc[19259](info): set up 2 sockets
Aug 12 01:13:12 dist-test-slave-tfpf krb5kdc[19259](info): commencing operation
krb5kdc: starting...
W20260812 01:13:13.273100 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:13 dist-test-slave-tfpf krb5kdc[19259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:13.413278 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:13.507241 19282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:13.507525 19282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:13.507570 19282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:13.508790 19282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:13.508876 19282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:13.508905 19282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:13.508989 19282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:13.509023 19282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:13.510715 19282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:13.511058 19282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:13.511333 19282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:13.516763 19288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:13.516769 19291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:13.516804 19289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:13.517025 19282 server_base.cc:1034] running on GCE node
I20260812 01:13:13.517421 19282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:13.517694 19282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:13.518869 19282 hybrid_clock.cc:648] HybridClock initialized: now 1786497193518854 us; error 36 us; skew 500 ppm
Aug 12 01:13:13 dist-test-slave-tfpf krb5kdc[19259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497193, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:13.521621 19282 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:13.522122 19282 webserver.cc:460] Webserver started at http://127.0.0.1:43009/ using document root <none> and password file <none>
I20260812 01:13:13.522429 19282 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:13.522480 19282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:13.522619 19282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:13.523924 19282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "48e7f4198b384f71b77a1c69d474d73c"
format_stamp: "Formatted at 2026-08-12 01:13:13 on dist-test-slave-tfpf"
server_key: "a2c1b0705b125c92267e12eac5c1cc22"
server_key_iv: "07b82ac0b8e90b5906d296c72641f6c1"
server_key_version: "encryptionkey@0"
I20260812 01:13:13.524482 19282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "48e7f4198b384f71b77a1c69d474d73c"
format_stamp: "Formatted at 2026-08-12 01:13:13 on dist-test-slave-tfpf"
server_key: "a2c1b0705b125c92267e12eac5c1cc22"
server_key_iv: "07b82ac0b8e90b5906d296c72641f6c1"
server_key_version: "encryptionkey@0"
I20260812 01:13:13.526345 19282 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260812 01:13:13.527271 19304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:13.527575 19282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:13.527689 19282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "48e7f4198b384f71b77a1c69d474d73c"
format_stamp: "Formatted at 2026-08-12 01:13:13 on dist-test-slave-tfpf"
server_key: "a2c1b0705b125c92267e12eac5c1cc22"
server_key_iv: "07b82ac0b8e90b5906d296c72641f6c1"
server_key_version: "encryptionkey@0"
I20260812 01:13:13.527776 19282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:13.540056 19282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:13.542734 19282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:13.542933 19282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:13.548274 19282 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42465
I20260812 01:13:13.548310 19382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42465 every 8 connection(s)
I20260812 01:13:13.548727 19282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:13.550108 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19282
I20260812 01:13:13.550155 19383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:13.550254 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:13.550489 17928 external_mini_cluster.cc:1428] Setting key 88eb9a5a713876b80c5438c0efebe608
I20260812 01:13:13.553901 19383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c: Bootstrap starting.
I20260812 01:13:13.554716 19383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:13.555102 19383 log.cc:824] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:13.555976 19383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c: No bootstrap required, opened a new log
I20260812 01:13:13.556785 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:13.551077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48031 (local address 127.0.0.1:42465)
0812 01:13:13.551277 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:13.551284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:13.551311 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:13:13.551788 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:13.551794 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:13.551842 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:13.551958 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:13.552354 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:13.553338 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:13.554516 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:13.555018 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:13.555122 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:13.556078 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:13.556091 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:13.556091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:13.556299 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:13.556332 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:13.556345 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:13.556452 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:13.556548 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 01:13:13.556581 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260812 01:13:13.558135 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:13.558478 19383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e7f4198b384f71b77a1c69d474d73c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42465 } }
I20260812 01:13:13.558634 19383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:13.558660 19383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48e7f4198b384f71b77a1c69d474d73c, State: Initialized, Role: FOLLOWER
I20260812 01:13:13.558823 19383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e7f4198b384f71b77a1c69d474d73c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42465 } }
I20260812 01:13:13.558895 19383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:13.558939 19383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:13.558996 19383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:13.559768 19383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e7f4198b384f71b77a1c69d474d73c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42465 } }
I20260812 01:13:13.559906 19383 leader_election.cc:302] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48e7f4198b384f71b77a1c69d474d73c; no voters:
I20260812 01:13:13.560118 19383 leader_election.cc:288] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:13.560173 19398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:13.560338 19398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [term 1 LEADER]: Becoming Leader. State: Replica: 48e7f4198b384f71b77a1c69d474d73c, State: Running, Role: LEADER
I20260812 01:13:13.560396 19383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:13.560521 19398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e7f4198b384f71b77a1c69d474d73c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42465 } }
I20260812 01:13:13.560933 19398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48e7f4198b384f71b77a1c69d474d73c. Latest consensus state: current_term: 1 leader_uuid: "48e7f4198b384f71b77a1c69d474d73c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e7f4198b384f71b77a1c69d474d73c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42465 } } }
I20260812 01:13:13.560948 19401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48e7f4198b384f71b77a1c69d474d73c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48e7f4198b384f71b77a1c69d474d73c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42465 } } }
I20260812 01:13:13.561110 19398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:13.561133 19401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c [sys.catalog]: This master's current role is: LEADER
W20260812 01:13:13.563256 19422 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:13.563443 19422 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:13.563517 19420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:13.563712 19420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:13.563870 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:13.559794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52825 (local address 127.0.0.1:42465)
0812 01:13:13.559836 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:13.559838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:13.559905 (+ 67us) server_negotiation.cc:407] Connection header received
0812 01:13:13.560072 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:13.560073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:13.560085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:13.560163 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:13.560392 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:13.561178 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:13.562209 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:13.562529 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:13.562574 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:13.563542 (+ 968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:13.563544 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:13.563544 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:13.563549 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:13.563566 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:13.563572 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:13.563635 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:13.563763 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 01:13:13.563773 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:13:13.565733 19420 catalog_manager.cc:1347] Generated new cluster ID: ab228e4ecde6482b80b176c0a2f823c2
I20260812 01:13:13.565810 19420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:13.593011 19420 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:13.593823 19420 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:13.618153 19420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48e7f4198b384f71b77a1c69d474d73c: Generated new TSK 0
I20260812 01:13:13.618400 19420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:13.632143 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19282
W20260812 01:13:13.648820 19250 connection.cc:535] client connection to 127.0.0.1:42465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42465): Connection reset by peer (error 104)
2026-08-12T01:13:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1675 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:13 dist-test-slave-tfpf krb5kdc[19430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:13 dist-test-slave-tfpf krb5kdc[19430](info): set up 2 sockets
Aug 12 01:13:13 dist-test-slave-tfpf krb5kdc[19430](info): commencing operation
krb5kdc: starting...
W20260812 01:13:14.955410 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:15 dist-test-slave-tfpf krb5kdc[19430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:15.098330 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:15.188812 19453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:15.189110 19453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:15.189167 19453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:15.190335 19453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:15.190407 19453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:15.190433 19453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:15.190464 19453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:15.190495 19453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:15.192088 19453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:15.192371 19453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:15.192610 19453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:15.198069 19461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:15.198122 19462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:15.198165 19467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:15.198264 19453 server_base.cc:1034] running on GCE node
I20260812 01:13:15.198606 19453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:15.198974 19453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:15.200167 19453 hybrid_clock.cc:648] HybridClock initialized: now 1786497195200154 us; error 51 us; skew 500 ppm
Aug 12 01:13:15 dist-test-slave-tfpf krb5kdc[19430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:15.207084 19453 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:15.207569 19453 webserver.cc:460] Webserver started at http://127.0.0.1:37299/ using document root <none> and password file <none>
I20260812 01:13:15.207821 19453 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:15.207890 19453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:15.208050 19453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:15.209259 19453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "aa423d61fd4c409db48c04318f27f62c"
format_stamp: "Formatted at 2026-08-12 01:13:15 on dist-test-slave-tfpf"
server_key: "95bd74cab70187d773ff183a8e0e1c79"
server_key_iv: "fff28f727e6efc95b643cc9fb8136ce1"
server_key_version: "encryptionkey@0"
I20260812 01:13:15.209650 19453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "aa423d61fd4c409db48c04318f27f62c"
format_stamp: "Formatted at 2026-08-12 01:13:15 on dist-test-slave-tfpf"
server_key: "95bd74cab70187d773ff183a8e0e1c79"
server_key_iv: "fff28f727e6efc95b643cc9fb8136ce1"
server_key_version: "encryptionkey@0"
I20260812 01:13:15.211114 19453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:13:15.212037 19479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:15.212302 19453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:15.212440 19453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "aa423d61fd4c409db48c04318f27f62c"
format_stamp: "Formatted at 2026-08-12 01:13:15 on dist-test-slave-tfpf"
server_key: "95bd74cab70187d773ff183a8e0e1c79"
server_key_iv: "fff28f727e6efc95b643cc9fb8136ce1"
server_key_version: "encryptionkey@0"
I20260812 01:13:15.212535 19453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:15.216717 19453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:15.219187 19453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:15.219364 19453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:15.223909 19453 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38375
I20260812 01:13:15.224058 19556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38375 every 8 connection(s)
I20260812 01:13:15.224509 19453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:15.225252 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19453
I20260812 01:13:15.225399 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:15.225607 17928 external_mini_cluster.cc:1428] Setting key bf975ee09d2badfd59d53210a4243653
I20260812 01:13:15.226118 19558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:15.228726 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:15.226243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48587 (local address 127.0.0.1:38375)
0812 01:13:15.226472 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:15.226490 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:15.226526 (+ 36us) server_negotiation.cc:407] Connection header received
0812 01:13:15.227150 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:15.227161 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:15.227287 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:15.227454 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:15.228208 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:15.228213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:15.228213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:15.228218 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:15.228269 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:15.228287 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:15.228369 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:15.228454 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 01:13:15.228511 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":112,"threads_started":1}
I20260812 01:13:15.230775 19558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c: Bootstrap starting.
I20260812 01:13:15.231755 19558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:15.232229 19558 log.cc:824] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:15.232847 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:15.233198 19558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c: No bootstrap required, opened a new log
I20260812 01:13:15.235967 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:15.234623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55813 (local address 127.0.0.1:38375)
0812 01:13:15.234687 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:15.234689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:15.234762 (+ 73us) server_negotiation.cc:407] Connection header received
0812 01:13:15.234830 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:15.234831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:15.234837 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:15.234895 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:15.235758 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:15.235759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:15.235760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:15.235764 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:15.235781 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:15.235786 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:15.235856 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:15.235894 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 01:13:15.235908 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 01:13:15.235885 19558 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa423d61fd4c409db48c04318f27f62c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260812 01:13:15.236071 19558 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:15.236112 19558 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa423d61fd4c409db48c04318f27f62c, State: Initialized, Role: FOLLOWER
I20260812 01:13:15.236248 19558 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa423d61fd4c409db48c04318f27f62c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260812 01:13:15.236351 19558 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:15.236384 19558 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:15.236444 19558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:15.237262 19558 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa423d61fd4c409db48c04318f27f62c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260812 01:13:15.237362 19558 leader_election.cc:302] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa423d61fd4c409db48c04318f27f62c; no voters:
I20260812 01:13:15.237495 19558 leader_election.cc:288] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:15.237586 19569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:15.237761 19558 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:15.237802 19569 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [term 1 LEADER]: Becoming Leader. State: Replica: aa423d61fd4c409db48c04318f27f62c, State: Running, Role: LEADER
I20260812 01:13:15.237936 19569 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa423d61fd4c409db48c04318f27f62c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260812 01:13:15.238451 19572 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa423d61fd4c409db48c04318f27f62c. Latest consensus state: current_term: 1 leader_uuid: "aa423d61fd4c409db48c04318f27f62c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa423d61fd4c409db48c04318f27f62c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } } }
I20260812 01:13:15.238556 19572 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:15.238828 19586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:15.238947 19570 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa423d61fd4c409db48c04318f27f62c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa423d61fd4c409db48c04318f27f62c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } } }
I20260812 01:13:15.239019 19570 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:15.239434 19586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:15.241196 19586 catalog_manager.cc:1347] Generated new cluster ID: 86efb3efd7894ea794a2f1262b852b60
I20260812 01:13:15.241277 19586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:15.248498 19586 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:15.248718 19586 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:15.259289 19586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa423d61fd4c409db48c04318f27f62c: Generated new TSK 0
I20260812 01:13:15.259529 19586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:15.264199 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19453
2026-08-12T01:13:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1628 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:15 dist-test-slave-tfpf krb5kdc[19611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:15 dist-test-slave-tfpf krb5kdc[19611](info): set up 2 sockets
Aug 12 01:13:15 dist-test-slave-tfpf krb5kdc[19611](info): commencing operation
krb5kdc: starting...
W20260812 01:13:16.634330 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:16 dist-test-slave-tfpf krb5kdc[19611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:16.781749 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:16.874197 19638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:16.874439 19638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:16.874465 19638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:16.875623 19638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:16.875669 19638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:16.875684 19638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:16.875695 19638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:16.875708 19638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:16.877316 19638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:16.877550 19638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:16.877746 19638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:16.883615 19653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:16.883620 19651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:16.884195 19638 server_base.cc:1034] running on GCE node
W20260812 01:13:16.883620 19648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:16.884598 19638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:16.885154 19638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:16.886394 19638 hybrid_clock.cc:648] HybridClock initialized: now 1786497196886353 us; error 55 us; skew 500 ppm
Aug 12 01:13:16 dist-test-slave-tfpf krb5kdc[19611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:16.889374 19638 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:16.889928 19638 webserver.cc:460] Webserver started at http://127.0.0.1:39779/ using document root <none> and password file <none>
I20260812 01:13:16.890273 19638 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:16.890372 19638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:16.890543 19638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:16.891741 19638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "03d4a1e27a65462cbbea3bc724e5b001"
format_stamp: "Formatted at 2026-08-12 01:13:16 on dist-test-slave-tfpf"
server_key: "7c22ad02f511e3130ac807edb27e4bff"
server_key_iv: "3e80b5b3d01c4b65ae827272ed1d5f31"
server_key_version: "encryptionkey@0"
I20260812 01:13:16.892141 19638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "03d4a1e27a65462cbbea3bc724e5b001"
format_stamp: "Formatted at 2026-08-12 01:13:16 on dist-test-slave-tfpf"
server_key: "7c22ad02f511e3130ac807edb27e4bff"
server_key_iv: "3e80b5b3d01c4b65ae827272ed1d5f31"
server_key_version: "encryptionkey@0"
I20260812 01:13:16.893688 19638 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:16.894558 19660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:16.894804 19638 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:13:16.894955 19638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "03d4a1e27a65462cbbea3bc724e5b001"
format_stamp: "Formatted at 2026-08-12 01:13:16 on dist-test-slave-tfpf"
server_key: "7c22ad02f511e3130ac807edb27e4bff"
server_key_iv: "3e80b5b3d01c4b65ae827272ed1d5f31"
server_key_version: "encryptionkey@0"
I20260812 01:13:16.895047 19638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:16.906428 19638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:16.908970 19638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:16.909188 19638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:16.913857 19638 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34911
I20260812 01:13:16.913899 19745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34911 every 8 connection(s)
I20260812 01:13:16.914319 19638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:16.915369 19747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:16.918171 19747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001: Bootstrap starting.
I20260812 01:13:16.918802 19747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:16.919173 19747 log.cc:824] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:16.919368 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19638
I20260812 01:13:16.919518 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:16.919747 17928 external_mini_cluster.cc:1428] Setting key 56088728df3bc93920e22dc7985461d5
I20260812 01:13:16.920318 19747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001: No bootstrap required, opened a new log
I20260812 01:13:16.922456 19747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d4a1e27a65462cbbea3bc724e5b001" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260812 01:13:16.922648 19747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:16.922696 19747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d4a1e27a65462cbbea3bc724e5b001, State: Initialized, Role: FOLLOWER
I20260812 01:13:16.922887 19747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d4a1e27a65462cbbea3bc724e5b001" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260812 01:13:16.923050 19747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:16.923098 19747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:16.923152 19747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:16.924363 19747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d4a1e27a65462cbbea3bc724e5b001" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260812 01:13:16.924471 19747 leader_election.cc:302] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03d4a1e27a65462cbbea3bc724e5b001; no voters:
I20260812 01:13:16.924736 19747 leader_election.cc:288] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:16.924820 19752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:16.925031 19752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [term 1 LEADER]: Becoming Leader. State: Replica: 03d4a1e27a65462cbbea3bc724e5b001, State: Running, Role: LEADER
I20260812 01:13:16.925110 19747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:16.925203 19752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d4a1e27a65462cbbea3bc724e5b001" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } }
I20260812 01:13:16.925637 19753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03d4a1e27a65462cbbea3bc724e5b001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d4a1e27a65462cbbea3bc724e5b001" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } } }
I20260812 01:13:16.925628 19752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03d4a1e27a65462cbbea3bc724e5b001. Latest consensus state: current_term: 1 leader_uuid: "03d4a1e27a65462cbbea3bc724e5b001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03d4a1e27a65462cbbea3bc724e5b001" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34911 } } }
I20260812 01:13:16.925796 19752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:16.925786 19753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:16.926070 19767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:16.926666 19767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 01:13:16 dist-test-slave-tfpf krb5kdc[19611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:13:16.930006 19767 catalog_manager.cc:1347] Generated new cluster ID: 1b9791036c074a178365d9398daa4fb3
I20260812 01:13:16.930069 19767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:16.934543 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:16.920456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44551 (local address 127.0.0.1:34911)
0812 01:13:16.920657 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:16.920665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:16.920692 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:13:16.922439 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:16.922448 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:16.922516 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:16.922690 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:16.923597 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:16.924642 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:16.925908 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:16.926702 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:16.926832 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:16.930350 (+ 3518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:16.930355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:16.930356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:16.930370 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:16.932620 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:16.933048 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:16.933051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:16.933052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:16.933248 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:16.933474 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:16.933475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:16.933476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:16.933837 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:16.934041 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:16.934122 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 01:13:16.934290 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260812 01:13:16.935698 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:16.945817 19751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:16.937406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42367 (local address 127.0.0.1:34911)
0812 01:13:16.937461 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:16.937462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:16.937775 (+ 313us) server_negotiation.cc:407] Connection header received
0812 01:13:16.937813 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:16.937814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:16.937826 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:16.937903 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:16.938536 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:16.939239 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:16.940239 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:16.940579 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:16.940689 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:16.942729 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:16.942731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:16.942731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:16.942745 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:16.944449 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:16.944903 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:16.944904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:16.944904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:16.944963 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:16.945244 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:16.945245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:16.945245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:16.945379 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:16.945492 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:16.945639 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:13:16.945720 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 01:13:16.952569 19767 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:16.953258 19767 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:16.959367 19767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03d4a1e27a65462cbbea3bc724e5b001: Generated new TSK 0
I20260812 01:13:16.959586 19767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:16.963830 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19638
2026-08-12T01:13:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1702 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:17 dist-test-slave-tfpf krb5kdc[19802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:17 dist-test-slave-tfpf krb5kdc[19802](info): set up 2 sockets
Aug 12 01:13:17 dist-test-slave-tfpf krb5kdc[19802](info): commencing operation
krb5kdc: starting...
W20260812 01:13:18.303274 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:18 dist-test-slave-tfpf krb5kdc[19802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:18.444449 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35043
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:18.536734 19821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:18.536976 19821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:18.537003 19821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:18.538239 19821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:18.538291 19821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:18.538305 19821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:18.538318 19821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:18.538331 19821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:18.539883 19821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:18.540099 19821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:18.540290 19821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:18.545488 19828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:18.545552 19827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:18.545845 19830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:18.546186 19821 server_base.cc:1034] running on GCE node
I20260812 01:13:18.546459 19821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:18.546764 19821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:18.547984 19821 hybrid_clock.cc:648] HybridClock initialized: now 1786497198547965 us; error 41 us; skew 500 ppm
Aug 12 01:13:18 dist-test-slave-tfpf krb5kdc[19802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:18.550875 19821 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:18.551352 19821 webserver.cc:460] Webserver started at http://127.0.0.1:41995/ using document root <none> and password file <none>
I20260812 01:13:18.551626 19821 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:18.551673 19821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:18.551807 19821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:18.552990 19821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "750b5ce480564fa7a497ff5b58365869"
format_stamp: "Formatted at 2026-08-12 01:13:18 on dist-test-slave-tfpf"
server_key: "77afc939908cd0113c50a7e1277192a6"
server_key_iv: "f8725229912647b92bf9afbe4bf4fd88"
server_key_version: "encryptionkey@0"
I20260812 01:13:18.553421 19821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "750b5ce480564fa7a497ff5b58365869"
format_stamp: "Formatted at 2026-08-12 01:13:18 on dist-test-slave-tfpf"
server_key: "77afc939908cd0113c50a7e1277192a6"
server_key_iv: "f8725229912647b92bf9afbe4bf4fd88"
server_key_version: "encryptionkey@0"
I20260812 01:13:18.554840 19821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:18.555694 19846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:18.556043 19821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:18.556146 19821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "750b5ce480564fa7a497ff5b58365869"
format_stamp: "Formatted at 2026-08-12 01:13:18 on dist-test-slave-tfpf"
server_key: "77afc939908cd0113c50a7e1277192a6"
server_key_iv: "f8725229912647b92bf9afbe4bf4fd88"
server_key_version: "encryptionkey@0"
I20260812 01:13:18.556273 19821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:18.571017 19821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:18.573568 19821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:18.573747 19821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:18.578092 19821 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35043
I20260812 01:13:18.578157 19948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35043 every 8 connection(s)
I20260812 01:13:18.578536 19821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:18.579520 19949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:18.581622 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 19821
I20260812 01:13:18.581750 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:18.581971 17928 external_mini_cluster.cc:1428] Setting key 5d85e313baa6fa3b167a8dcb0d5bb88c
I20260812 01:13:18.582399 19949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869: Bootstrap starting.
I20260812 01:13:18.583148 19949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:18.583461 19949 log.cc:824] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:18.584185 19949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869: No bootstrap required, opened a new log
I20260812 01:13:18.586123 19949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "750b5ce480564fa7a497ff5b58365869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35043 } }
I20260812 01:13:18.586295 19949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:18.586345 19949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 750b5ce480564fa7a497ff5b58365869, State: Initialized, Role: FOLLOWER
I20260812 01:13:18.586516 19949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "750b5ce480564fa7a497ff5b58365869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35043 } }
I20260812 01:13:18.586601 19949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:18.586625 19949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:18.586687 19949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:18.587386 19949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "750b5ce480564fa7a497ff5b58365869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35043 } }
I20260812 01:13:18.587517 19949 leader_election.cc:302] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 750b5ce480564fa7a497ff5b58365869; no voters:
I20260812 01:13:18.587734 19949 leader_election.cc:288] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:18.587771 19956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:18.587903 19956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [term 1 LEADER]: Becoming Leader. State: Replica: 750b5ce480564fa7a497ff5b58365869, State: Running, Role: LEADER
I20260812 01:13:18.587987 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:18.582576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32971 (local address 127.0.0.1:35043)
0812 01:13:18.582788 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:18.582796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:18.582822 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:13:18.583356 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:18.583364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:18.583414 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:18.583566 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:18.583810 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:18.584776 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:18.585673 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:18.586085 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:18.586229 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:18.587013 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:18.587017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:18.587017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:18.587216 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:18.587250 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:18.587262 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:18.587319 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:18.587436 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 01:13:18.587686 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260812 01:13:18.588028 19949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:18.588059 19956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "750b5ce480564fa7a497ff5b58365869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35043 } }
I20260812 01:13:18.589322 19957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "750b5ce480564fa7a497ff5b58365869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "750b5ce480564fa7a497ff5b58365869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35043 } } }
I20260812 01:13:18.589421 19957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:18.589658 19958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 750b5ce480564fa7a497ff5b58365869. Latest consensus state: current_term: 1 leader_uuid: "750b5ce480564fa7a497ff5b58365869" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "750b5ce480564fa7a497ff5b58365869" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35043 } } }
I20260812 01:13:18.589797 19958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:18.589850 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 01:13:18.591514 19975 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:18.591581 19975 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:18.591675 19976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:18.591925 19976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:18.593662 19976 catalog_manager.cc:1347] Generated new cluster ID: edc133f172a841a6be800c7669247932
I20260812 01:13:18.593753 19976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:18.596550 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:18.591332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57135 (local address 127.0.0.1:35043)
0812 01:13:18.591373 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:18.591375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:18.593256 (+ 1881us) server_negotiation.cc:407] Connection header received
0812 01:13:18.593320 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:18.593321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:18.593334 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:18.593389 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:18.593568 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:18.594292 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:18.595194 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:18.595492 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:18.595552 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:18.596308 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:18.596309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:18.596309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:18.596314 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:18.596330 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:18.596334 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:18.596384 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:18.596455 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 01:13:18.596476 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:13:18.617417 19976 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:18.618024 19976 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:18.621492 19976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 750b5ce480564fa7a497ff5b58365869: Generated new TSK 0
I20260812 01:13:18.621696 19976 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:18.624926 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 19821
2026-08-12T01:13:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1666 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:18 dist-test-slave-tfpf krb5kdc[19993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:18 dist-test-slave-tfpf krb5kdc[19993](info): set up 2 sockets
Aug 12 01:13:18 dist-test-slave-tfpf krb5kdc[19993](info): commencing operation
krb5kdc: starting...
W20260812 01:13:19.967520 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:20 dist-test-slave-tfpf krb5kdc[19993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:20.109623 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:20.200847 20011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:20.201107 20011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:20.201133 20011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:20.202306 20011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:20.202353 20011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:20.202368 20011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:20.202380 20011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:20.202394 20011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:20.203923 20011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:20.204155 20011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:20.204351 20011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:20.209916 20022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:20.209911 20020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:20.210072 20011 server_base.cc:1034] running on GCE node
W20260812 01:13:20.210180 20017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:20.210495 20011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:20.210878 20011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:20.212098 20011 hybrid_clock.cc:648] HybridClock initialized: now 1786497200212052 us; error 60 us; skew 500 ppm
Aug 12 01:13:20 dist-test-slave-tfpf krb5kdc[19993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:20.220731 20011 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:20.221272 20011 webserver.cc:460] Webserver started at http://127.0.0.1:33069/ using document root <none> and password file <none>
I20260812 01:13:20.221527 20011 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:20.221580 20011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:20.221722 20011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:20.222859 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "596fc26395cb4f5d8af172d6760ef41e"
format_stamp: "Formatted at 2026-08-12 01:13:20 on dist-test-slave-tfpf"
server_key: "ea0d4e0d2e66dbb419ff8ddc65407d6c"
server_key_iv: "469fd6053dcd2dbc193484ddd5274bc8"
server_key_version: "encryptionkey@0"
I20260812 01:13:20.223260 20011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "596fc26395cb4f5d8af172d6760ef41e"
format_stamp: "Formatted at 2026-08-12 01:13:20 on dist-test-slave-tfpf"
server_key: "ea0d4e0d2e66dbb419ff8ddc65407d6c"
server_key_iv: "469fd6053dcd2dbc193484ddd5274bc8"
server_key_version: "encryptionkey@0"
I20260812 01:13:20.224747 20011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:20.225723 20032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:20.225983 20011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:20.226114 20011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "596fc26395cb4f5d8af172d6760ef41e"
format_stamp: "Formatted at 2026-08-12 01:13:20 on dist-test-slave-tfpf"
server_key: "ea0d4e0d2e66dbb419ff8ddc65407d6c"
server_key_iv: "469fd6053dcd2dbc193484ddd5274bc8"
server_key_version: "encryptionkey@0"
I20260812 01:13:20.226191 20011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:20.229749 20011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:20.231812 20011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:20.231974 20011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:20.236534 20011 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37279
I20260812 01:13:20.236591 20116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37279 every 8 connection(s)
I20260812 01:13:20.236979 20011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:20.238157 20117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:20.240566 20117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e: Bootstrap starting.
I20260812 01:13:20.241292 20117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:20.241600 20117 log.cc:824] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:20.242453 20117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e: No bootstrap required, opened a new log
I20260812 01:13:20.244156 20117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596fc26395cb4f5d8af172d6760ef41e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37279 } }
I20260812 01:13:20.244306 20117 raft_consensus.cc:374] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:20.244330 20117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596fc26395cb4f5d8af172d6760ef41e, State: Initialized, Role: FOLLOWER
I20260812 01:13:20.244494 20117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596fc26395cb4f5d8af172d6760ef41e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37279 } }
I20260812 01:13:20.244561 20117 raft_consensus.cc:388] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:20.244582 20117 raft_consensus.cc:482] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:20.244637 20117 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:20.245289 20117 raft_consensus.cc:504] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596fc26395cb4f5d8af172d6760ef41e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37279 } }
I20260812 01:13:20.245383 20117 leader_election.cc:302] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 596fc26395cb4f5d8af172d6760ef41e; no voters:
I20260812 01:13:20.245613 20117 leader_election.cc:288] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:20.245692 20124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:20.246135 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20011
I20260812 01:13:20.246237 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:20.246451 17928 external_mini_cluster.cc:1428] Setting key c0276427044cf19e33d5a7f64f6a5746
I20260812 01:13:20.245898 20124 raft_consensus.cc:686] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [term 1 LEADER]: Becoming Leader. State: Replica: 596fc26395cb4f5d8af172d6760ef41e, State: Running, Role: LEADER
I20260812 01:13:20.245949 20117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:20.246673 20124 consensus_queue.cc:237] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596fc26395cb4f5d8af172d6760ef41e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37279 } }
I20260812 01:13:20.247511 20125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "596fc26395cb4f5d8af172d6760ef41e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596fc26395cb4f5d8af172d6760ef41e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37279 } } }
I20260812 01:13:20.247673 20126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 596fc26395cb4f5d8af172d6760ef41e. Latest consensus state: current_term: 1 leader_uuid: "596fc26395cb4f5d8af172d6760ef41e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "596fc26395cb4f5d8af172d6760ef41e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37279 } } }
I20260812 01:13:20.247779 20126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:20.247726 20125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:20.248239 20147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:20.248855 20147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:20.249199 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:20.247229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44693 (local address 127.0.0.1:37279)
0812 01:13:20.247426 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:20.247435 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:20.247473 (+ 38us) server_negotiation.cc:407] Connection header received
0812 01:13:20.247598 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:20.247606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:20.247642 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:20.248093 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:20.248662 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:20.248666 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:20.248667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:20.248672 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:20.248725 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:20.248744 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:20.248839 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:20.248932 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 01:13:20.248978 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260812 01:13:20.250708 20147 catalog_manager.cc:1347] Generated new cluster ID: f18637c2655f4168a66a49e5444841c7
I20260812 01:13:20.250723 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:20.250761 20147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:20.258901 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:20.257589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42317 (local address 127.0.0.1:37279)
0812 01:13:20.257637 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:20.257639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:20.257788 (+ 149us) server_negotiation.cc:407] Connection header received
0812 01:13:20.257863 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:20.257864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:20.257870 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:20.257920 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:20.258611 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:20.258612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:20.258612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:20.258617 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:20.258633 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:20.258638 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:20.258709 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:20.258794 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 01:13:20.258810 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:13:20.261417 20147 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:20.261605 20147 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:20.272228 20147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 596fc26395cb4f5d8af172d6760ef41e: Generated new TSK 0
I20260812 01:13:20.272472 20147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:20.279469 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20011
2026-08-12T01:13:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:20 dist-test-slave-tfpf krb5kdc[20175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:20 dist-test-slave-tfpf krb5kdc[20175](info): set up 2 sockets
Aug 12 01:13:20 dist-test-slave-tfpf krb5kdc[20175](info): commencing operation
krb5kdc: starting...
W20260812 01:13:21.654242 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:21 dist-test-slave-tfpf krb5kdc[20175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:21.805166 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:21.900902 20195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:21.901248 20195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:21.901281 20195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:21.902535 20195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:21.902591 20195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:21.902607 20195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:21.902621 20195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:21.902632 20195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:21.904299 20195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:21.904592 20195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:21.904847 20195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:21.910377 20202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:21.910362 20204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:21.910763 20195 server_base.cc:1034] running on GCE node
W20260812 01:13:21.910867 20201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:21.911188 20195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:21.911533 20195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:21.912721 20195 hybrid_clock.cc:648] HybridClock initialized: now 1786497201912699 us; error 37 us; skew 500 ppm
Aug 12 01:13:21 dist-test-slave-tfpf krb5kdc[20175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:21.915699 20195 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:21.916275 20195 webserver.cc:460] Webserver started at http://127.0.0.1:46717/ using document root <none> and password file <none>
I20260812 01:13:21.916520 20195 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:21.916592 20195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:21.916778 20195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:21.918006 20195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34"
format_stamp: "Formatted at 2026-08-12 01:13:21 on dist-test-slave-tfpf"
server_key: "3485629858b43b397c387e1fa229e1fa"
server_key_iv: "b6afc3b01139e7fe43a98af95216c982"
server_key_version: "encryptionkey@0"
I20260812 01:13:21.918397 20195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34"
format_stamp: "Formatted at 2026-08-12 01:13:21 on dist-test-slave-tfpf"
server_key: "3485629858b43b397c387e1fa229e1fa"
server_key_iv: "b6afc3b01139e7fe43a98af95216c982"
server_key_version: "encryptionkey@0"
I20260812 01:13:21.919904 20195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 01:13:21.921145 20213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:21.921377 20195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:21.921514 20195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34"
format_stamp: "Formatted at 2026-08-12 01:13:21 on dist-test-slave-tfpf"
server_key: "3485629858b43b397c387e1fa229e1fa"
server_key_iv: "b6afc3b01139e7fe43a98af95216c982"
server_key_version: "encryptionkey@0"
I20260812 01:13:21.921612 20195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:21.930190 20195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:21.932757 20195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:21.932979 20195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:21.938124 20195 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38205
I20260812 01:13:21.938184 20294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38205 every 8 connection(s)
I20260812 01:13:21.938619 20195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:21.939656 20296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:21.942577 20296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34: Bootstrap starting.
I20260812 01:13:21.942652 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20195
I20260812 01:13:21.942761 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:21.942961 17928 external_mini_cluster.cc:1428] Setting key 1eaf48b2729e1113561254358803cbd0
I20260812 01:13:21.943492 20296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:21.944164 20296 log.cc:824] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:21.945415 20296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34: No bootstrap required, opened a new log
I20260812 01:13:21.948046 20296 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260812 01:13:21.948249 20296 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:21.948287 20296 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b04eb38c5ca45e4ad1e4d23ea5caa34, State: Initialized, Role: FOLLOWER
I20260812 01:13:21.948417 20296 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260812 01:13:21.948498 20296 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:21.948532 20296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:21.948572 20296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:21.949501 20296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260812 01:13:21.949640 20296 leader_election.cc:302] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b04eb38c5ca45e4ad1e4d23ea5caa34; no voters:
I20260812 01:13:21.949885 20296 leader_election.cc:288] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:21.949911 20304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:21.950062 20304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [term 1 LEADER]: Becoming Leader. State: Replica: 3b04eb38c5ca45e4ad1e4d23ea5caa34, State: Running, Role: LEADER
I20260812 01:13:21.950194 20296 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:21.950213 20304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
Aug 12 01:13:21 dist-test-slave-tfpf krb5kdc[20175](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:13:21.952243 20306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } } }
I20260812 01:13:21.952368 20306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [sys.catalog]: This master's current role is: LEADER
W20260812 01:13:21.953042 20325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 01:13:21.952236 20307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b04eb38c5ca45e4ad1e4d23ea5caa34. Latest consensus state: current_term: 1 leader_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b04eb38c5ca45e4ad1e4d23ea5caa34" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } } }
W20260812 01:13:21.953130 20325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:21.953188 20307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:21.953330 20329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:21.953567 20329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:21.955777 20329 catalog_manager.cc:1347] Generated new cluster ID: d67780e3a83544b69719eeb69487d838
I20260812 01:13:21.955857 20329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:21.957625 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:21.943735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:35771 (local address 127.0.0.1:38205)
0812 01:13:21.943930 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:21.943942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:21.943984 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:13:21.944686 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:21.944695 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:21.944751 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:21.944882 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:21.945737 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:21.947160 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:21.948014 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:21.948670 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:21.948794 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:21.952291 (+ 3497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:21.952297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:21.952297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:21.952313 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:21.954629 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:21.955606 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:21.955610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:21.955610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:21.955767 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:21.956089 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:21.956090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:21.956091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:21.956398 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:21.956555 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:21.957268 (+ 713us) server_negotiation.cc:299] Negotiation successful
0812 01:13:21.957394 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260812 01:13:21.958984 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:21.965570 20329 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:21.966388 20329 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:21.969125 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:21.960855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:54421 (local address 127.0.0.1:38205)
0812 01:13:21.960931 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:21.960933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:21.960942 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:21.961053 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:21.961054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:21.961097 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:21.961150 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:21.961829 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:21.962615 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:21.963670 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:21.963998 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:21.964043 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:21.965833 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:21.965835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:21.965835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:21.965843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:21.967708 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:21.968200 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:21.968201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:21.968201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:21.968261 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:21.968475 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:21.968476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:21.968477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:21.968598 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:21.968688 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:21.968918 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 01:13:21.968994 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 01:13:21.973728 20329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b04eb38c5ca45e4ad1e4d23ea5caa34: Generated new TSK 0
I20260812 01:13:21.973970 20329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:21.978312 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20195
W20260812 01:13:22.011842 20171 connection.cc:535] client connection to 127.0.0.1:38205 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38205): Connection reset by peer (error 104)
2026-08-12T01:13:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1709 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:22 dist-test-slave-tfpf krb5kdc[20342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:22 dist-test-slave-tfpf krb5kdc[20342](info): set up 2 sockets
Aug 12 01:13:22 dist-test-slave-tfpf krb5kdc[20342](info): commencing operation
krb5kdc: starting...
W20260812 01:13:23.340957 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:23 dist-test-slave-tfpf krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:23.488631 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:23.584653 20363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:23.584930 20363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:23.584975 20363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:23.586185 20363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:23.586266 20363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:23.586292 20363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:23.586323 20363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:23.586354 20363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:23.587935 20363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:23.588248 20363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:23.588510 20363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:23.593779 20371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:23.593827 20376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:23.593779 20374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:23.594135 20363 server_base.cc:1034] running on GCE node
I20260812 01:13:23.594415 20363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:23.594722 20363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:23.595958 20363 hybrid_clock.cc:648] HybridClock initialized: now 1786497203595944 us; error 30 us; skew 500 ppm
Aug 12 01:13:23 dist-test-slave-tfpf krb5kdc[20342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497203, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:23.598985 20363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:23.599484 20363 webserver.cc:460] Webserver started at http://127.0.0.1:37849/ using document root <none> and password file <none>
I20260812 01:13:23.599768 20363 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:23.599843 20363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:23.600008 20363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:23.601193 20363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "841348bdc5554c9f8f47a0782e656469"
format_stamp: "Formatted at 2026-08-12 01:13:23 on dist-test-slave-tfpf"
server_key: "f99a8a6619fdb591a4e44979893e273a"
server_key_iv: "b86e5e1702177402596dc4fd4362cf4c"
server_key_version: "encryptionkey@0"
I20260812 01:13:23.601568 20363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "841348bdc5554c9f8f47a0782e656469"
format_stamp: "Formatted at 2026-08-12 01:13:23 on dist-test-slave-tfpf"
server_key: "f99a8a6619fdb591a4e44979893e273a"
server_key_iv: "b86e5e1702177402596dc4fd4362cf4c"
server_key_version: "encryptionkey@0"
I20260812 01:13:23.603238 20363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:13:23.604298 20384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:23.604686 20363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:23.604965 20363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "841348bdc5554c9f8f47a0782e656469"
format_stamp: "Formatted at 2026-08-12 01:13:23 on dist-test-slave-tfpf"
server_key: "f99a8a6619fdb591a4e44979893e273a"
server_key_iv: "b86e5e1702177402596dc4fd4362cf4c"
server_key_version: "encryptionkey@0"
I20260812 01:13:23.605566 20363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:23.630308 20363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:23.633030 20363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:23.633263 20363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:23.638387 20363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38923
I20260812 01:13:23.638433 20467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38923 every 8 connection(s)
I20260812 01:13:23.638818 20363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:23.640158 20468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:23.642578 20468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469: Bootstrap starting.
I20260812 01:13:23.643203 20468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:23.643448 20468 log.cc:824] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:23.644199 20468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469: No bootstrap required, opened a new log
I20260812 01:13:23.645953 20468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841348bdc5554c9f8f47a0782e656469" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38923 } }
I20260812 01:13:23.646118 20468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:23.646150 20468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 841348bdc5554c9f8f47a0782e656469, State: Initialized, Role: FOLLOWER
I20260812 01:13:23.646206 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20363
I20260812 01:13:23.646342 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:23.646315 20468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841348bdc5554c9f8f47a0782e656469" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38923 } }
I20260812 01:13:23.646399 20468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:23.646458 20468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:23.646510 20468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:23.646624 17928 external_mini_cluster.cc:1428] Setting key d3b0a04c33d79fbb8ece6353a3140d10
I20260812 01:13:23.647377 20468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841348bdc5554c9f8f47a0782e656469" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38923 } }
I20260812 01:13:23.647495 20468 leader_election.cc:302] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 841348bdc5554c9f8f47a0782e656469; no voters:
I20260812 01:13:23.647706 20468 leader_election.cc:288] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:23.647730 20478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:23.647845 20478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [term 1 LEADER]: Becoming Leader. State: Replica: 841348bdc5554c9f8f47a0782e656469, State: Running, Role: LEADER
I20260812 01:13:23.647989 20468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:23.647984 20478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841348bdc5554c9f8f47a0782e656469" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38923 } }
I20260812 01:13:23.648488 20483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 841348bdc5554c9f8f47a0782e656469. Latest consensus state: current_term: 1 leader_uuid: "841348bdc5554c9f8f47a0782e656469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841348bdc5554c9f8f47a0782e656469" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38923 } } }
I20260812 01:13:23.648586 20483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:23.648821 20482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "841348bdc5554c9f8f47a0782e656469" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "841348bdc5554c9f8f47a0782e656469" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38923 } } }
I20260812 01:13:23.649267 20488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:23.649339 20482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:23.649981 20488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:23.651597 20488 catalog_manager.cc:1347] Generated new cluster ID: a5d4d91bcea645309b368d099eca8505
I20260812 01:13:23.651641 20488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:23.653285 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:23.647230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:37087 (local address 127.0.0.1:38923)
0812 01:13:23.647433 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:23.647442 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:23.647469 (+ 27us) server_negotiation.cc:407] Connection header received
0812 01:13:23.647638 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:23.647645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:23.647701 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:23.648251 (+ 550us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:23.648662 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:23.650120 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:23.651091 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:23.651689 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:23.651800 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:23.652564 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:23.652568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:23.652569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:23.652773 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:23.652806 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:23.652817 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:23.652915 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:23.652998 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 01:13:23.653027 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260812 01:13:23.654174 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:23.660301 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:23.655591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:56823 (local address 127.0.0.1:38923)
0812 01:13:23.655632 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:23.655634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:23.657334 (+ 1700us) server_negotiation.cc:407] Connection header received
0812 01:13:23.657414 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:23.657416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:23.657426 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:23.657476 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:23.657620 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:23.658140 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:23.658893 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:23.659135 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:23.659233 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:23.660006 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:23.660007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:23.660008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:23.660013 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:23.660030 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:23.660034 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:23.660094 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:23.660170 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 01:13:23.660186 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:13:23.674361 20488 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:23.674952 20488 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:23.677999 20488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 841348bdc5554c9f8f47a0782e656469: Generated new TSK 0
I20260812 01:13:23.678228 20488 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:23.682621 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20363
W20260812 01:13:23.702694 20340 connection.cc:535] client connection to 127.0.0.1:38923 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38923): Connection reset by peer (error 104)
2026-08-12T01:13:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1689 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:23 dist-test-slave-tfpf krb5kdc[20514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:23 dist-test-slave-tfpf krb5kdc[20514](info): set up 2 sockets
Aug 12 01:13:23 dist-test-slave-tfpf krb5kdc[20514](info): commencing operation
krb5kdc: starting...
W20260812 01:13:25.027104 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:25 dist-test-slave-tfpf krb5kdc[20514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:25.168345 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:25.258232 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:25.258524 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:25.258569 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:25.259750 20534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:25.259825 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:25.259851 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:25.259881 20534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:25.259913 20534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:25.261500 20534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:25.261816 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:25.262063 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:25.267542 20547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:25.267573 20549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:25.267566 20546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:25.267887 20534 server_base.cc:1034] running on GCE node
I20260812 01:13:25.268152 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:25.268422 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:25.269603 20534 hybrid_clock.cc:648] HybridClock initialized: now 1786497205269578 us; error 34 us; skew 500 ppm
Aug 12 01:13:25 dist-test-slave-tfpf krb5kdc[20514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497205, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:25.277704 20534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:25.278254 20534 webserver.cc:460] Webserver started at http://127.0.0.1:34533/ using document root <none> and password file <none>
I20260812 01:13:25.278462 20534 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:25.278503 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:25.278610 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:25.279696 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "65c6b6728a56474bb59dbfc5686d4f14"
format_stamp: "Formatted at 2026-08-12 01:13:25 on dist-test-slave-tfpf"
server_key: "6f913c8c2e5e1f93a6c2b4ed69988e7b"
server_key_iv: "35c818fb7cf4588fbee58b5ad485d048"
server_key_version: "encryptionkey@0"
I20260812 01:13:25.280014 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "65c6b6728a56474bb59dbfc5686d4f14"
format_stamp: "Formatted at 2026-08-12 01:13:25 on dist-test-slave-tfpf"
server_key: "6f913c8c2e5e1f93a6c2b4ed69988e7b"
server_key_iv: "35c818fb7cf4588fbee58b5ad485d048"
server_key_version: "encryptionkey@0"
I20260812 01:13:25.281445 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:25.282316 20564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:25.282549 20534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:25.282675 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "65c6b6728a56474bb59dbfc5686d4f14"
format_stamp: "Formatted at 2026-08-12 01:13:25 on dist-test-slave-tfpf"
server_key: "6f913c8c2e5e1f93a6c2b4ed69988e7b"
server_key_iv: "35c818fb7cf4588fbee58b5ad485d048"
server_key_version: "encryptionkey@0"
I20260812 01:13:25.282764 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:25.286630 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:25.288976 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:25.289311 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:25.293915 20534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34109
I20260812 01:13:25.293972 20647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34109 every 8 connection(s)
I20260812 01:13:25.294353 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:25.295094 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20534
I20260812 01:13:25.295233 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:25.295431 17928 external_mini_cluster.cc:1428] Setting key 45bb16a6047435b98ce89ec743b2a451
I20260812 01:13:25.295657 20649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:25.298341 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:25.296056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:46913 (local address 127.0.0.1:34109)
0812 01:13:25.296289 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:25.296299 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:25.296334 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:13:25.296925 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:25.296934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:25.296989 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:25.297176 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:25.297849 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:25.297853 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:25.297853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:25.297858 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:25.297906 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:25.297920 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:25.298008 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:25.298111 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 01:13:25.298155 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260812 01:13:25.299450 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:25.299543 20649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14: Bootstrap starting.
I20260812 01:13:25.300213 20649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:25.300598 20649 log.cc:824] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:25.301533 20649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14: No bootstrap required, opened a new log
I20260812 01:13:25.302251 20654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:25.300961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:51281 (local address 127.0.0.1:34109)
0812 01:13:25.301003 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:25.301005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:25.301144 (+ 139us) server_negotiation.cc:407] Connection header received
0812 01:13:25.301218 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:25.301219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:25.301224 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:25.301295 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:25.301957 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:25.301958 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:25.301958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:25.301963 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:25.301978 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:25.301982 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:25.302049 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:25.302178 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 01:13:25.302192 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:13:25.303781 20649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c6b6728a56474bb59dbfc5686d4f14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260812 01:13:25.303933 20649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:25.303958 20649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c6b6728a56474bb59dbfc5686d4f14, State: Initialized, Role: FOLLOWER
I20260812 01:13:25.304075 20649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c6b6728a56474bb59dbfc5686d4f14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260812 01:13:25.304165 20649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:25.304190 20649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:25.304220 20649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:25.304775 20649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c6b6728a56474bb59dbfc5686d4f14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260812 01:13:25.304867 20649 leader_election.cc:302] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65c6b6728a56474bb59dbfc5686d4f14; no voters:
I20260812 01:13:25.305019 20649 leader_election.cc:288] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:25.305056 20664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:25.305245 20664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [term 1 LEADER]: Becoming Leader. State: Replica: 65c6b6728a56474bb59dbfc5686d4f14, State: Running, Role: LEADER
I20260812 01:13:25.305331 20649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:25.305361 20664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c6b6728a56474bb59dbfc5686d4f14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } }
I20260812 01:13:25.305886 20666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65c6b6728a56474bb59dbfc5686d4f14. Latest consensus state: current_term: 1 leader_uuid: "65c6b6728a56474bb59dbfc5686d4f14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c6b6728a56474bb59dbfc5686d4f14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } } }
I20260812 01:13:25.305932 20665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65c6b6728a56474bb59dbfc5686d4f14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65c6b6728a56474bb59dbfc5686d4f14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34109 } } }
I20260812 01:13:25.306000 20666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:25.306034 20665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:25.306258 20677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:25.306983 20677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:25.308961 20677 catalog_manager.cc:1347] Generated new cluster ID: bd5c4e44da2f4eb2b4fe870873147760
I20260812 01:13:25.309026 20677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:25.339464 20677 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:25.340193 20677 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:25.350065 20677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65c6b6728a56474bb59dbfc5686d4f14: Generated new TSK 0
I20260812 01:13:25.350332 20677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:25.356187 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20534
2026-08-12T01:13:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:25 dist-test-slave-tfpf krb5kdc[20698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:25 dist-test-slave-tfpf krb5kdc[20698](info): set up 2 sockets
Aug 12 01:13:25 dist-test-slave-tfpf krb5kdc[20698](info): commencing operation
krb5kdc: starting...
W20260812 01:13:26.672055 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:26 dist-test-slave-tfpf krb5kdc[20698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:26.808785 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:26.898059 20716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:26.898350 20716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:26.898394 20716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:26.899569 20716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:26.899648 20716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:26.899675 20716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:26.899708 20716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:26.899737 20716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:26.901336 20716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:26.901644 20716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:26.901891 20716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:26.907449 20727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:26.907485 20729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:26.907644 20716 server_base.cc:1034] running on GCE node
W20260812 01:13:26.907454 20723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:26.908067 20716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:26.908352 20716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:26.909523 20716 hybrid_clock.cc:648] HybridClock initialized: now 1786497206909489 us; error 44 us; skew 500 ppm
Aug 12 01:13:26 dist-test-slave-tfpf krb5kdc[20698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:26.912213 20716 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:26.912721 20716 webserver.cc:460] Webserver started at http://127.0.0.1:34569/ using document root <none> and password file <none>
I20260812 01:13:26.912940 20716 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:26.912983 20716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:26.913154 20716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:26.914222 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "b8a1738a26e74502889871248dfabb6d"
format_stamp: "Formatted at 2026-08-12 01:13:26 on dist-test-slave-tfpf"
server_key: "6482b296ea5754fff8f9f53a4f483aa1"
server_key_iv: "ec94b4f5baf774acc56f9ef2424804c8"
server_key_version: "encryptionkey@0"
I20260812 01:13:26.914549 20716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "b8a1738a26e74502889871248dfabb6d"
format_stamp: "Formatted at 2026-08-12 01:13:26 on dist-test-slave-tfpf"
server_key: "6482b296ea5754fff8f9f53a4f483aa1"
server_key_iv: "ec94b4f5baf774acc56f9ef2424804c8"
server_key_version: "encryptionkey@0"
I20260812 01:13:26.915896 20716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:13:26.916738 20739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:26.916939 20716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:26.917178 20716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "b8a1738a26e74502889871248dfabb6d"
format_stamp: "Formatted at 2026-08-12 01:13:26 on dist-test-slave-tfpf"
server_key: "6482b296ea5754fff8f9f53a4f483aa1"
server_key_iv: "ec94b4f5baf774acc56f9ef2424804c8"
server_key_version: "encryptionkey@0"
I20260812 01:13:26.917279 20716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:26.937103 20716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:26.939724 20716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:26.939931 20716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:26.944834 20716 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32821
I20260812 01:13:26.944883 20828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32821 every 8 connection(s)
I20260812 01:13:26.945319 20716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:26.945724 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20716
I20260812 01:13:26.945880 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:26.946141 17928 external_mini_cluster.cc:1428] Setting key 4ea898bcc07d7ed5d2d3df106562108b
I20260812 01:13:26.946561 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:26.949677 20829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d: Bootstrap starting.
I20260812 01:13:26.950330 20829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:26.950641 20829 log.cc:824] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:26.951610 20829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d: No bootstrap required, opened a new log
Aug 12 01:13:26 dist-test-slave-tfpf krb5kdc[20698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 01:13:26.954382 20829 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a1738a26e74502889871248dfabb6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260812 01:13:26.954571 20829 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:26.954627 20829 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a1738a26e74502889871248dfabb6d, State: Initialized, Role: FOLLOWER
I20260812 01:13:26.954782 20829 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a1738a26e74502889871248dfabb6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260812 01:13:26.954885 20829 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:26.954923 20829 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:26.954986 20829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:26.955829 20829 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a1738a26e74502889871248dfabb6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260812 01:13:26.955952 20829 leader_election.cc:302] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8a1738a26e74502889871248dfabb6d; no voters:
I20260812 01:13:26.956158 20829 leader_election.cc:288] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:26.956276 20838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:26.956471 20829 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:26.956480 20838 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [term 1 LEADER]: Becoming Leader. State: Replica: b8a1738a26e74502889871248dfabb6d, State: Running, Role: LEADER
I20260812 01:13:26.956632 20838 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a1738a26e74502889871248dfabb6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260812 01:13:26.957161 20838 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8a1738a26e74502889871248dfabb6d. Latest consensus state: current_term: 1 leader_uuid: "b8a1738a26e74502889871248dfabb6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a1738a26e74502889871248dfabb6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260812 01:13:26.957264 20838 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:26.957196 20839 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8a1738a26e74502889871248dfabb6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a1738a26e74502889871248dfabb6d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260812 01:13:26.957427 20839 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:26.957495 20845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:26.958673 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:26.946750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:43281 (local address 127.0.0.1:32821)
0812 01:13:26.946988 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:26.946997 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:26.947025 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:13:26.947692 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:26.947700 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:26.947755 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:26.947913 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:26.948606 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:26.949603 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:26.950408 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:26.950904 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:26.951155 (+ 251us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:26.954013 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:26.954019 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:26.954019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:26.954036 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:26.956248 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:26.956808 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:26.956811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:26.956812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:26.956924 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:26.957648 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:26.957649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:26.957650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:26.957954 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:26.958158 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:26.958252 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 01:13:26.958359 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260812 01:13:26.959218 20845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:26.960054 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:26.961858 20845 catalog_manager.cc:1347] Generated new cluster ID: 5a420c2937e64ed58f4086ff26db507f
I20260812 01:13:26.961916 20845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:26.976346 20845 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:26.977239 20845 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:26.979946 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:26.961370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:59065 (local address 127.0.0.1:32821)
0812 01:13:26.961411 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:26.961413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:26.969318 (+ 7905us) server_negotiation.cc:407] Connection header received
0812 01:13:26.969412 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:26.969414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:26.969428 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:26.969528 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:26.970641 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:26.971665 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:26.973053 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:26.973551 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:26.973623 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:26.976333 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:26.976335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:26.976335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:26.976344 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:26.978553 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:26.979066 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:26.979067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:26.979067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:26.979128 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:26.979374 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:26.979374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:26.979375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:26.979496 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:26.979595 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:26.979769 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 01:13:26.979844 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:13:26.986366 20845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8a1738a26e74502889871248dfabb6d: Generated new TSK 0
I20260812 01:13:26.986641 20845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:26.989363 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20716
W20260812 01:13:27.014258 20696 connection.cc:535] client connection to 127.0.0.1:32821 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32821): Connection reset by peer (error 104)
2026-08-12T01:13:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1642 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:27 dist-test-slave-tfpf krb5kdc[20875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:27 dist-test-slave-tfpf krb5kdc[20875](info): set up 2 sockets
Aug 12 01:13:27 dist-test-slave-tfpf krb5kdc[20875](info): commencing operation
krb5kdc: starting...
W20260812 01:13:28.504375 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.476s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:28 dist-test-slave-tfpf krb5kdc[20875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:28.657390 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:28.753791 20892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:28.754051 20892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:28.754079 20892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:28.755378 20892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:28.755438 20892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:28.755455 20892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:28.755467 20892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:28.755479 20892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:28.757759 20892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:28.758154 20892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:28.758476 20892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:28.764554 20898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:28.764547 20902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:28.764902 20899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:28.765312 20892 server_base.cc:1034] running on GCE node
I20260812 01:13:28.765789 20892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:28.766281 20892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:28.767488 20892 hybrid_clock.cc:648] HybridClock initialized: now 1786497208767439 us; error 69 us; skew 500 ppm
Aug 12 01:13:28 dist-test-slave-tfpf krb5kdc[20875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:28.770660 20892 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:28.771260 20892 webserver.cc:460] Webserver started at http://127.0.0.1:41429/ using document root <none> and password file <none>
I20260812 01:13:28.771559 20892 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:28.771633 20892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:28.771799 20892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:28.773012 20892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "2cd8d57e3b80464d852d24918431bc56"
format_stamp: "Formatted at 2026-08-12 01:13:28 on dist-test-slave-tfpf"
server_key: "86ec5c5e78001999f278d0ad1acb34c0"
server_key_iv: "d5183cbe789faecb1a1c0a35bd95a371"
server_key_version: "encryptionkey@0"
I20260812 01:13:28.773693 20892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "2cd8d57e3b80464d852d24918431bc56"
format_stamp: "Formatted at 2026-08-12 01:13:28 on dist-test-slave-tfpf"
server_key: "86ec5c5e78001999f278d0ad1acb34c0"
server_key_iv: "d5183cbe789faecb1a1c0a35bd95a371"
server_key_version: "encryptionkey@0"
I20260812 01:13:28.775404 20892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:13:28.776381 20910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:28.776877 20892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:28.777184 20892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "2cd8d57e3b80464d852d24918431bc56"
format_stamp: "Formatted at 2026-08-12 01:13:28 on dist-test-slave-tfpf"
server_key: "86ec5c5e78001999f278d0ad1acb34c0"
server_key_iv: "d5183cbe789faecb1a1c0a35bd95a371"
server_key_version: "encryptionkey@0"
I20260812 01:13:28.777295 20892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:28.800786 20892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:28.803213 20892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:28.803409 20892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:28.808326 20892 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40417
I20260812 01:13:28.808388 20999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40417 every 8 connection(s)
I20260812 01:13:28.808837 20892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:28.809965 21000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:28.812463 21000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56: Bootstrap starting.
I20260812 01:13:28.813145 21000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:28.813454 21000 log.cc:824] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:28.814321 21000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56: No bootstrap required, opened a new log
I20260812 01:13:28.815521 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 20892
I20260812 01:13:28.815641 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:28.815862 17928 external_mini_cluster.cc:1428] Setting key acc67674522a33b3d852fa8730e11eea
I20260812 01:13:28.816561 21000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd8d57e3b80464d852d24918431bc56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 01:13:28.816787 21000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:28.816831 21000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd8d57e3b80464d852d24918431bc56, State: Initialized, Role: FOLLOWER
I20260812 01:13:28.816960 21000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd8d57e3b80464d852d24918431bc56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 01:13:28.817046 21000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:28.817117 21000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:28.817242 21000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:28.818274 21000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd8d57e3b80464d852d24918431bc56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 01:13:28.818398 21000 leader_election.cc:302] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cd8d57e3b80464d852d24918431bc56; no voters:
I20260812 01:13:28.818559 21000 leader_election.cc:288] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:28.818620 21010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:28.818823 21000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:28.819317 21010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd8d57e3b80464d852d24918431bc56, State: Running, Role: LEADER
I20260812 01:13:28.819541 21010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd8d57e3b80464d852d24918431bc56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } }
I20260812 01:13:28.819954 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cd8d57e3b80464d852d24918431bc56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd8d57e3b80464d852d24918431bc56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } } }
I20260812 01:13:28.820091 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:28.820370 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cd8d57e3b80464d852d24918431bc56. Latest consensus state: current_term: 1 leader_uuid: "2cd8d57e3b80464d852d24918431bc56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cd8d57e3b80464d852d24918431bc56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40417 } } }
I20260812 01:13:28.820497 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56 [sys.catalog]: This master's current role is: LEADER
W20260812 01:13:28.820863 21029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:28.820928 21029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:28.821020 21031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:28.821269 21031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:28.823096 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:28.816509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:33119 (local address 127.0.0.1:40417)
0812 01:13:28.816767 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:28.816778 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:28.816815 (+ 37us) server_negotiation.cc:407] Connection header received
0812 01:13:28.816942 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:28.816949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:28.817010 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:28.817190 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:28.817482 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:28.819326 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:28.820579 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:28.821355 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:28.821471 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:28.822321 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:28.822325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:28.822325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:28.822550 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:28.822596 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:28.822608 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:28.822716 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:28.822819 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 01:13:28.822850 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":116,"threads_started":1}
I20260812 01:13:28.823113 21031 catalog_manager.cc:1347] Generated new cluster ID: d8a9ee06fab34cdf9a360ce9155c3d3a
I20260812 01:13:28.823182 21031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:28.826043 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:28.830860 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:28.827594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:34535 (local address 127.0.0.1:40417)
0812 01:13:28.827633 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:28.827635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:28.827821 (+ 186us) server_negotiation.cc:407] Connection header received
0812 01:13:28.827880 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:28.827882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:28.827904 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:28.827960 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:28.828161 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:28.828663 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:28.829523 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:28.829771 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:28.829850 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:28.830600 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:28.830602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:28.830602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:28.830607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:28.830622 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:28.830627 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:28.830682 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:28.830742 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:13:28.830754 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:13:28.834430 21031 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:28.835156 21031 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:28.840744 21031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cd8d57e3b80464d852d24918431bc56: Generated new TSK 0
I20260812 01:13:28.841202 21031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:28.846786 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 20892
W20260812 01:13:28.876619 20873 connection.cc:535] client connection to 127.0.0.1:40417 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40417): Connection reset by peer (error 104)
2026-08-12T01:13:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1862 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:28 dist-test-slave-tfpf krb5kdc[21047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:28 dist-test-slave-tfpf krb5kdc[21047](info): set up 2 sockets
Aug 12 01:13:28 dist-test-slave-tfpf krb5kdc[21047](info): commencing operation
krb5kdc: starting...
W20260812 01:13:30.208679 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:30 dist-test-slave-tfpf krb5kdc[21047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497210, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 01:13:30.352687 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:30.440101 21063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:30.440410 21063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:30.440452 21063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:30.441710 21063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:30.441795 21063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:30.441821 21063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:30.441845 21063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:30.441876 21063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:30.443470 21063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:30.443780 21063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:30.444037 21063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:30.449328 21075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:30.449328 21069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:30.449427 21070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:30.449608 21063 server_base.cc:1034] running on GCE node
I20260812 01:13:30.449946 21063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:30.450273 21063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:30.451467 21063 hybrid_clock.cc:648] HybridClock initialized: now 1786497210451449 us; error 35 us; skew 500 ppm
Aug 12 01:13:30 dist-test-slave-tfpf krb5kdc[21047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497210, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:30.459734 21063 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 01:13:30.460271 21063 webserver.cc:460] Webserver started at http://127.0.0.1:42001/ using document root <none> and password file <none>
I20260812 01:13:30.460516 21063 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:30.460563 21063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:30.460712 21063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:30.461930 21063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "43f6ac540210426797cd57bb1873fe67"
format_stamp: "Formatted at 2026-08-12 01:13:30 on dist-test-slave-tfpf"
server_key: "152a92f00b0ccfaee97e05473f3744c1"
server_key_iv: "a3dc2b21ada57cd5fc0a87ce4e4f5b09"
server_key_version: "encryptionkey@0"
I20260812 01:13:30.462324 21063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "43f6ac540210426797cd57bb1873fe67"
format_stamp: "Formatted at 2026-08-12 01:13:30 on dist-test-slave-tfpf"
server_key: "152a92f00b0ccfaee97e05473f3744c1"
server_key_iv: "a3dc2b21ada57cd5fc0a87ce4e4f5b09"
server_key_version: "encryptionkey@0"
I20260812 01:13:30.463855 21063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:13:30.465034 21089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:30.465358 21063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:30.465461 21063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "43f6ac540210426797cd57bb1873fe67"
format_stamp: "Formatted at 2026-08-12 01:13:30 on dist-test-slave-tfpf"
server_key: "152a92f00b0ccfaee97e05473f3744c1"
server_key_iv: "a3dc2b21ada57cd5fc0a87ce4e4f5b09"
server_key_version: "encryptionkey@0"
I20260812 01:13:30.465580 21063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:30.469704 21063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:30.473285 21063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:30.473466 21063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:30.478981 21063 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38773
I20260812 01:13:30.479156 21174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38773 every 8 connection(s)
I20260812 01:13:30.479548 21063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:30.479732 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21063
I20260812 01:13:30.479876 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:30.480084 17928 external_mini_cluster.cc:1428] Setting key 3f00b8da2126e584c3542f6d151d6eeb
I20260812 01:13:30.480921 21175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:30.483007 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:30.480692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:53443 (local address 127.0.0.1:38773)
0812 01:13:30.480896 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:30.480904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:30.480930 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:13:30.481437 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:30.481445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:30.481475 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:30.481611 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:30.482458 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:30.482461 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:30.482462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:30.482466 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:30.482513 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:30.482525 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:30.482598 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:30.482693 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 01:13:30.482732 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260812 01:13:30.484397 21175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67: Bootstrap starting.
I20260812 01:13:30.484426 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:30.485185 21175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:30.485607 21175 log.cc:824] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:30.486639 21175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67: No bootstrap required, opened a new log
I20260812 01:13:30.487988 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:30.486512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.74:42689 (local address 127.0.0.1:38773)
0812 01:13:30.486550 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:30.486552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:30.486700 (+ 148us) server_negotiation.cc:407] Connection header received
0812 01:13:30.486807 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:30.486808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:30.486814 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:30.486867 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:30.487568 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:30.487570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:30.487570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:30.487574 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:30.487589 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:30.487594 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:30.487648 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:30.487755 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 01:13:30.487768 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:13:30.489459 21175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f6ac540210426797cd57bb1873fe67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260812 01:13:30.489650 21175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:30.489702 21175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43f6ac540210426797cd57bb1873fe67, State: Initialized, Role: FOLLOWER
I20260812 01:13:30.489840 21175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f6ac540210426797cd57bb1873fe67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260812 01:13:30.489960 21175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:30.490008 21175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:30.490062 21175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:30.490715 21175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f6ac540210426797cd57bb1873fe67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260812 01:13:30.490846 21175 leader_election.cc:302] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43f6ac540210426797cd57bb1873fe67; no voters:
I20260812 01:13:30.491067 21175 leader_election.cc:288] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:30.491127 21190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:30.491331 21190 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [term 1 LEADER]: Becoming Leader. State: Replica: 43f6ac540210426797cd57bb1873fe67, State: Running, Role: LEADER
I20260812 01:13:30.491379 21175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:30.491544 21190 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f6ac540210426797cd57bb1873fe67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } }
I20260812 01:13:30.492157 21191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43f6ac540210426797cd57bb1873fe67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f6ac540210426797cd57bb1873fe67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } } }
I20260812 01:13:30.492165 21192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43f6ac540210426797cd57bb1873fe67. Latest consensus state: current_term: 1 leader_uuid: "43f6ac540210426797cd57bb1873fe67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43f6ac540210426797cd57bb1873fe67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38773 } } }
I20260812 01:13:30.492271 21191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:30.492292 21192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:30.492617 21198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:30.493245 21198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:30.495657 21198 catalog_manager.cc:1347] Generated new cluster ID: 43f4092433da44738479587d9d4df7ac
I20260812 01:13:30.495726 21198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:30.504297 21198 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:30.504509 21198 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:30.513206 21198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43f6ac540210426797cd57bb1873fe67: Generated new TSK 0
I20260812 01:13:30.513597 21198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:30.522709 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21063
2026-08-12T01:13:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1657 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:30 dist-test-slave-tfpf krb5kdc[21219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:30 dist-test-slave-tfpf krb5kdc[21219](info): set up 2 sockets
Aug 12 01:13:30 dist-test-slave-tfpf krb5kdc[21219](info): commencing operation
krb5kdc: starting...
W20260812 01:13:31.872642 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:31 dist-test-slave-tfpf krb5kdc[21219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:32.008628 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33001
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:33001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:32.097785 21240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:32.098417 21240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:32.098487 21240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:32.099881 21240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:32.099968 21240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:32.099997 21240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:32.100262 21240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:32.100333 21240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:32.102633 21240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:33001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:32.103209 21240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:32.103503 21240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:32.108932 21251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:32.108950 21254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:32.108981 21249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:32.109223 21240 server_base.cc:1034] running on GCE node
I20260812 01:13:32.109668 21240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:32.110015 21240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:32.111181 21240 hybrid_clock.cc:648] HybridClock initialized: now 1786497212111170 us; error 33 us; skew 500 ppm
Aug 12 01:13:32 dist-test-slave-tfpf krb5kdc[21219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497212, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:32.113906 21240 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:32.114343 21240 webserver.cc:460] Webserver started at http://127.17.130.62:46795/ using document root <none> and password file <none>
I20260812 01:13:32.114601 21240 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:32.114653 21240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:32.114789 21240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:32.115912 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "11d44f821420497fba812b0da06c249a"
format_stamp: "Formatted at 2026-08-12 01:13:32 on dist-test-slave-tfpf"
server_key: "2099c9007cde793d5bd60b7831d22a37"
server_key_iv: "3a635a853d23a5408ce282ea8e9cd2bb"
server_key_version: "encryptionkey@0"
I20260812 01:13:32.116317 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "11d44f821420497fba812b0da06c249a"
format_stamp: "Formatted at 2026-08-12 01:13:32 on dist-test-slave-tfpf"
server_key: "2099c9007cde793d5bd60b7831d22a37"
server_key_iv: "3a635a853d23a5408ce282ea8e9cd2bb"
server_key_version: "encryptionkey@0"
I20260812 01:13:32.117888 21240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:13:32.118782 21267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:32.119184 21240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:32.119397 21240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "11d44f821420497fba812b0da06c249a"
format_stamp: "Formatted at 2026-08-12 01:13:32 on dist-test-slave-tfpf"
server_key: "2099c9007cde793d5bd60b7831d22a37"
server_key_iv: "3a635a853d23a5408ce282ea8e9cd2bb"
server_key_version: "encryptionkey@0"
I20260812 01:13:32.119493 21240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:32.136586 21240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:32.139461 21240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:32.139664 21240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:32.145092 21240 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:33001
I20260812 01:13:32.145133 21364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:33001 every 8 connection(s)
I20260812 01:13:32.145630 21240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:32.146792 21365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:32.149271 21365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a: Bootstrap starting.
I20260812 01:13:32.149880 21365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:32.150156 21365 log.cc:824] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:32.150974 21365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a: No bootstrap required, opened a new log
I20260812 01:13:32.152681 21365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d44f821420497fba812b0da06c249a" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33001 } }
I20260812 01:13:32.152846 21365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:32.152872 21365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11d44f821420497fba812b0da06c249a, State: Initialized, Role: FOLLOWER
I20260812 01:13:32.153047 21365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d44f821420497fba812b0da06c249a" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33001 } }
I20260812 01:13:32.153190 21365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:32.153218 21365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:32.153251 21365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:32.154235 21365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d44f821420497fba812b0da06c249a" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33001 } }
I20260812 01:13:32.154338 21365 leader_election.cc:302] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11d44f821420497fba812b0da06c249a; no voters:
I20260812 01:13:32.154490 21365 leader_election.cc:288] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:32.154587 21371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:32.154790 21371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [term 1 LEADER]: Becoming Leader. State: Replica: 11d44f821420497fba812b0da06c249a, State: Running, Role: LEADER
I20260812 01:13:32.154820 21365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:32.154886 21371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d44f821420497fba812b0da06c249a" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33001 } }
I20260812 01:13:32.155334 21375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11d44f821420497fba812b0da06c249a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d44f821420497fba812b0da06c249a" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33001 } } }
I20260812 01:13:32.155439 21375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:32.155418 21376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11d44f821420497fba812b0da06c249a. Latest consensus state: current_term: 1 leader_uuid: "11d44f821420497fba812b0da06c249a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11d44f821420497fba812b0da06c249a" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33001 } } }
I20260812 01:13:32.155500 21376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:32.155620 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21240
I20260812 01:13:32.155771 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:32.155943 17928 external_mini_cluster.cc:1428] Setting key 0ab3e32a56f4531771fc21521bf8001d
I20260812 01:13:32.156307 21382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:32.157311 21382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:32.159183 21382 catalog_manager.cc:1347] Generated new cluster ID: 6936aa43a195454cb6f4246b4c6c0d12
I20260812 01:13:32.159242 21382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:13:32 dist-test-slave-tfpf krb5kdc[21219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:32.169202 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:32.156488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50781 (local address 127.17.130.62:33001)
0812 01:13:32.156869 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:32.156880 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:32.156919 (+ 39us) server_negotiation.cc:407] Connection header received
0812 01:13:32.157055 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:32.157082 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:32.157143 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:32.158127 (+ 984us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:32.158229 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:32.159572 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:32.160663 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:32.161370 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:32.161507 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:32.164813 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:32.164824 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:32.164824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:32.164854 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:32.167254 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:32.167759 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:32.167764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:32.167764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:32.167900 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:32.168155 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:32.168156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:32.168156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:32.168493 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:32.168664 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:32.168773 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 01:13:32.168885 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":75,"threads_started":1}
I20260812 01:13:32.170424 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:32.178413 21382 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:32.179273 21382 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:32.180676 21393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:32.171945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55543 (local address 127.17.130.62:33001)
0812 01:13:32.172005 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:32.172007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:32.172205 (+ 198us) server_negotiation.cc:407] Connection header received
0812 01:13:32.172260 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:32.172261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:32.172271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:32.172325 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:32.172972 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:32.173758 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:32.174817 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:32.175136 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:32.175197 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:32.176873 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:32.176875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:32.176875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:32.176882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:32.179158 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:32.179673 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:32.179675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:32.179675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:32.179752 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:32.180036 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:32.180037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:32.180037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:32.180188 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:32.180329 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:32.180488 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 01:13:32.180570 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:13:32.183413 21382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11d44f821420497fba812b0da06c249a: Generated new TSK 0
I20260812 01:13:32.183640 21382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:32.189640 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21240
W20260812 01:13:32.202358 21217 connection.cc:535] client connection to 127.17.130.62:33001 recv error: Network error: failed to read from TLS socket (remote: 127.17.130.62:33001): Connection reset by peer (error 104)
2026-08-12T01:13:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1668 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:32 dist-test-slave-tfpf krb5kdc[21414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:32 dist-test-slave-tfpf krb5kdc[21414](info): set up 2 sockets
Aug 12 01:13:32 dist-test-slave-tfpf krb5kdc[21414](info): commencing operation
krb5kdc: starting...
W20260812 01:13:33.517660 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:33 dist-test-slave-tfpf krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:33.658696 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34041
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:34041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:33.755968 21432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:33.756254 21432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:33.756299 21432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:33.757728 21432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:33.757807 21432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:33.757833 21432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:33.757865 21432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:33.757901 21432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:33.760048 21432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:34041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:34041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:33.760419 21432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:33.760710 21432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:33.766538 21443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:33.766541 21446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:33.766541 21442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:33.767560 21432 server_base.cc:1034] running on GCE node
I20260812 01:13:33.767943 21432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:33.768297 21432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:33.769511 21432 hybrid_clock.cc:648] HybridClock initialized: now 1786497213769477 us; error 57 us; skew 500 ppm
Aug 12 01:13:33 dist-test-slave-tfpf krb5kdc[21414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497213, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:33.772469 21432 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:33.773013 21432 webserver.cc:460] Webserver started at http://127.17.130.62:44911/ using document root <none> and password file <none>
I20260812 01:13:33.773356 21432 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:33.773414 21432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:33.773545 21432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:33.774724 21432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "8d99c2b95cf54cb9af75050d4ed5caeb"
format_stamp: "Formatted at 2026-08-12 01:13:33 on dist-test-slave-tfpf"
server_key: "781a3cf551acb4b410823fc30f2bff58"
server_key_iv: "198b5445b327b07a7406ad8a45e6a770"
server_key_version: "encryptionkey@0"
I20260812 01:13:33.775117 21432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "8d99c2b95cf54cb9af75050d4ed5caeb"
format_stamp: "Formatted at 2026-08-12 01:13:33 on dist-test-slave-tfpf"
server_key: "781a3cf551acb4b410823fc30f2bff58"
server_key_iv: "198b5445b327b07a7406ad8a45e6a770"
server_key_version: "encryptionkey@0"
I20260812 01:13:33.776829 21432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 01:13:33.777850 21460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:33.778162 21432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:33.778290 21432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "8d99c2b95cf54cb9af75050d4ed5caeb"
format_stamp: "Formatted at 2026-08-12 01:13:33 on dist-test-slave-tfpf"
server_key: "781a3cf551acb4b410823fc30f2bff58"
server_key_iv: "198b5445b327b07a7406ad8a45e6a770"
server_key_version: "encryptionkey@0"
I20260812 01:13:33.778370 21432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:33.794684 21432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:33.797432 21432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:33.797631 21432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:33.803476 21432 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:34041
I20260812 01:13:33.803514 21545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:34041 every 8 connection(s)
I20260812 01:13:33.803917 21432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:33.804884 21547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:33.806257 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21432
I20260812 01:13:33.806407 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:33.806605 17928 external_mini_cluster.cc:1428] Setting key 523016df7b869e9e3aa815e92501d572
I20260812 01:13:33.808488 21547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb: Bootstrap starting.
I20260812 01:13:33.809253 21547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:33.809593 21547 log.cc:824] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:33.810382 21547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb: No bootstrap required, opened a new log
I20260812 01:13:33.812270 21547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34041 } }
I20260812 01:13:33.812451 21547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:33.812520 21547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d99c2b95cf54cb9af75050d4ed5caeb, State: Initialized, Role: FOLLOWER
I20260812 01:13:33.812711 21547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34041 } }
I20260812 01:13:33.812767 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:33.807272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36691 (local address 127.17.130.62:34041)
0812 01:13:33.807460 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:33.807468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:33.807801 (+ 333us) server_negotiation.cc:407] Connection header received
0812 01:13:33.808259 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:33.808267 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:33.808323 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:33.808448 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:33.808714 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:33.809790 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:33.810669 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:33.811085 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:33.811205 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:33.812078 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:33.812082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:33.812082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:33.812254 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:33.812286 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:33.812298 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:33.812393 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:33.812479 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 01:13:33.812509 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260812 01:13:33.812790 21547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:33.812969 21547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:33.813144 21547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:33.814026 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:33.814191 21547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34041 } }
I20260812 01:13:33.814324 21547 leader_election.cc:302] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d99c2b95cf54cb9af75050d4ed5caeb; no voters:
I20260812 01:13:33.814539 21547 leader_election.cc:288] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:33.814567 21558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:33.814682 21558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [term 1 LEADER]: Becoming Leader. State: Replica: 8d99c2b95cf54cb9af75050d4ed5caeb, State: Running, Role: LEADER
I20260812 01:13:33.814828 21558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34041 } }
I20260812 01:13:33.814834 21547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:33.815441 21560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d99c2b95cf54cb9af75050d4ed5caeb. Latest consensus state: current_term: 1 leader_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34041 } } }
I20260812 01:13:33.815533 21560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:33.815758 21559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d99c2b95cf54cb9af75050d4ed5caeb" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 34041 } } }
I20260812 01:13:33.815865 21559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:33.816607 21570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:33.817440 21570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:33.819748 21570 catalog_manager.cc:1347] Generated new cluster ID: 23720da9b96143dd956eae52e47563b8
I20260812 01:13:33.819815 21570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:33.820986 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:33.815719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58173 (local address 127.17.130.62:34041)
0812 01:13:33.815762 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:33.815764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:33.816462 (+ 698us) server_negotiation.cc:407] Connection header received
0812 01:13:33.816515 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:33.816516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:33.816536 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:33.816596 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:33.816874 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:33.817879 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:33.819477 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:33.819732 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:33.819816 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:33.820698 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:33.820699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:33.820700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:33.820705 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:33.820718 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:33.820723 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:33.820751 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:33.820898 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 01:13:33.820910 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:13:33.826354 21570 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:33.826869 21570 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:33.832350 21570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d99c2b95cf54cb9af75050d4ed5caeb: Generated new TSK 0
I20260812 01:13:33.832576 21570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:33.837675 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21432
W20260812 01:13:33.862596 21412 connection.cc:535] client connection to 127.17.130.62:34041 recv error: Network error: failed to read from TLS socket (remote: 127.17.130.62:34041): Connection reset by peer (error 104)
2026-08-12T01:13:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1660 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:33 dist-test-slave-tfpf krb5kdc[21600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:33 dist-test-slave-tfpf krb5kdc[21600](info): set up 2 sockets
Aug 12 01:13:33 dist-test-slave-tfpf krb5kdc[21600](info): commencing operation
krb5kdc: starting...
W20260812 01:13:35.193483 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:35 dist-test-slave-tfpf krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:35.332646 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33371
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:42701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:33371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:35.423090 21618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:35.423346 21618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:35.423372 21618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:35.424490 21618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:35.424530 21618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:35.424546 21618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:35.424559 21618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:35.424571 21618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:35.426107 21618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:42701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:33371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:35.426352 21618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:35.426540 21618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:35.431911 21625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:35.431919 21624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:35.431919 21627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:35.432597 21618 server_base.cc:1034] running on GCE node
I20260812 01:13:35.432868 21618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:35.435480 21618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:35.436717 21618 hybrid_clock.cc:648] HybridClock initialized: now 1786497215436689 us; error 45 us; skew 500 ppm
Aug 12 01:13:35 dist-test-slave-tfpf krb5kdc[21600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497215, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:35.439663 21618 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:35.440147 21618 webserver.cc:460] Webserver started at http://127.17.130.62:38653/ using document root <none> and password file <none>
I20260812 01:13:35.440398 21618 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:35.440476 21618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:35.440624 21618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:35.441823 21618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "65b3f646acff46dfa56034d39695c8d9"
format_stamp: "Formatted at 2026-08-12 01:13:35 on dist-test-slave-tfpf"
server_key: "57b556e2f53fb06ad99eff21557e51f9"
server_key_iv: "0523304629d782dfa055b8d607439cf9"
server_key_version: "encryptionkey@0"
I20260812 01:13:35.442237 21618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "65b3f646acff46dfa56034d39695c8d9"
format_stamp: "Formatted at 2026-08-12 01:13:35 on dist-test-slave-tfpf"
server_key: "57b556e2f53fb06ad99eff21557e51f9"
server_key_iv: "0523304629d782dfa055b8d607439cf9"
server_key_version: "encryptionkey@0"
I20260812 01:13:35.443766 21618 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 01:13:35.444768 21642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:35.445272 21618 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:35.445497 21618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "65b3f646acff46dfa56034d39695c8d9"
format_stamp: "Formatted at 2026-08-12 01:13:35 on dist-test-slave-tfpf"
server_key: "57b556e2f53fb06ad99eff21557e51f9"
server_key_iv: "0523304629d782dfa055b8d607439cf9"
server_key_version: "encryptionkey@0"
I20260812 01:13:35.445623 21618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:35.449613 21618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:35.452682 21618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:35.452873 21618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:35.458230 21618 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:33371
I20260812 01:13:35.458426 21714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:33371 every 8 connection(s)
I20260812 01:13:35.458734 21618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:35.459448 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21618
I20260812 01:13:35.459599 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:35.459810 17928 external_mini_cluster.cc:1428] Setting key 7d9f7cc8df159a40f3b4d50b7f547bd3
I20260812 01:13:35.460062 21716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:35.463620 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9: Bootstrap starting.
I20260812 01:13:35.464285 21716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:35.464560 21716 log.cc:824] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:35.465355 21716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9: No bootstrap required, opened a new log
I20260812 01:13:35.466773 21716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b3f646acff46dfa56034d39695c8d9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33371 } }
I20260812 01:13:35.466929 21716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:35.466954 21716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b3f646acff46dfa56034d39695c8d9, State: Initialized, Role: FOLLOWER
I20260812 01:13:35.467110 21716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b3f646acff46dfa56034d39695c8d9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33371 } }
I20260812 01:13:35.467228 21716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:35.467267 21716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:35.467321 21716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:35.468067 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:35.460659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54543 (local address 127.17.130.62:33371)
0812 01:13:35.460894 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:35.460905 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:35.460938 (+ 33us) server_negotiation.cc:407] Connection header received
0812 01:13:35.461753 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:35.461762 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:35.461797 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:35.461963 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:35.467446 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:35.467451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:35.467451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:35.467457 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:35.467512 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:35.467524 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:35.467591 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:35.467748 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 01:13:35.467824 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
I20260812 01:13:35.468600 21716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b3f646acff46dfa56034d39695c8d9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33371 } }
I20260812 01:13:35.468742 21716 leader_election.cc:302] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65b3f646acff46dfa56034d39695c8d9; no voters:
I20260812 01:13:35.468938 21716 leader_election.cc:288] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:35.468972 21723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:35.469152 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:35.469208 21723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [term 1 LEADER]: Becoming Leader. State: Replica: 65b3f646acff46dfa56034d39695c8d9, State: Running, Role: LEADER
I20260812 01:13:35.469260 21716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:35.469296 21723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b3f646acff46dfa56034d39695c8d9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33371 } }
I20260812 01:13:35.469640 21723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65b3f646acff46dfa56034d39695c8d9. Latest consensus state: current_term: 1 leader_uuid: "65b3f646acff46dfa56034d39695c8d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b3f646acff46dfa56034d39695c8d9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33371 } } }
I20260812 01:13:35.469722 21723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:35.469647 21727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65b3f646acff46dfa56034d39695c8d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65b3f646acff46dfa56034d39695c8d9" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33371 } } }
I20260812 01:13:35.469776 21727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9 [sys.catalog]: This master's current role is: LEADER
W20260812 01:13:35.471388 21752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:35.471524 21752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:35.471652 21732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:35.471904 21732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:35.472143 21721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:35.470929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59977 (local address 127.17.130.62:33371)
0812 01:13:35.470969 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:35.470970 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:35.471027 (+ 57us) server_negotiation.cc:407] Connection header received
0812 01:13:35.471147 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:35.471148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:35.471155 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:35.471224 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:35.471896 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:35.471897 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:35.471897 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:35.471902 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:35.471918 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:35.471923 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:35.471990 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:35.472074 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 01:13:35.472089 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:13:35.473647 21732 catalog_manager.cc:1347] Generated new cluster ID: c39cc784c8ff40b29b62b6dfe2feec89
I20260812 01:13:35.473721 21732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:35.484643 21732 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:35.484874 21732 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:35.494293 21732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65b3f646acff46dfa56034d39695c8d9: Generated new TSK 0
I20260812 01:13:35.494542 21732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:35.498481 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21618
2026-08-12T01:13:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1655 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:35 dist-test-slave-tfpf krb5kdc[21761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:35 dist-test-slave-tfpf krb5kdc[21761](info): set up 2 sockets
Aug 12 01:13:35 dist-test-slave-tfpf krb5kdc[21761](info): commencing operation
krb5kdc: starting...
W20260812 01:13:36.836664 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:36 dist-test-slave-tfpf krb5kdc[21761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:36.980268 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:36357
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:37.071501 21779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:37.071727 21779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:37.071753 21779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:37.072860 21779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:37.072903 21779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:37.072917 21779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:37.072930 21779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:37.072942 21779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:37.074522 21779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:37.074757 21779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:37.074958 21779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:37.080442 21789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:37.080489 21788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:37.080696 21792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:37.080587 21779 server_base.cc:1034] running on GCE node
I20260812 01:13:37.081106 21779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:37.081569 21779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:37.082747 21779 hybrid_clock.cc:648] HybridClock initialized: now 1786497217082724 us; error 45 us; skew 500 ppm
Aug 12 01:13:37 dist-test-slave-tfpf krb5kdc[21761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497217, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:37.085644 21779 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:37.086179 21779 webserver.cc:460] Webserver started at http://127.17.130.62:39383/ using document root <none> and password file <none>
I20260812 01:13:37.086460 21779 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:37.086535 21779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:37.086695 21779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:37.087843 21779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "9c8ac4c852b94dd89bde4353c64970ca"
format_stamp: "Formatted at 2026-08-12 01:13:37 on dist-test-slave-tfpf"
server_key: "09fc488c53a77d0941b697e6c481724a"
server_key_iv: "3b2dc774d0632fa02fd1b95c8f14d0c3"
server_key_version: "encryptionkey@0"
I20260812 01:13:37.088241 21779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "9c8ac4c852b94dd89bde4353c64970ca"
format_stamp: "Formatted at 2026-08-12 01:13:37 on dist-test-slave-tfpf"
server_key: "09fc488c53a77d0941b697e6c481724a"
server_key_iv: "3b2dc774d0632fa02fd1b95c8f14d0c3"
server_key_version: "encryptionkey@0"
I20260812 01:13:37.089812 21779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 01:13:37.090741 21802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:37.091073 21779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:37.091202 21779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "9c8ac4c852b94dd89bde4353c64970ca"
format_stamp: "Formatted at 2026-08-12 01:13:37 on dist-test-slave-tfpf"
server_key: "09fc488c53a77d0941b697e6c481724a"
server_key_iv: "3b2dc774d0632fa02fd1b95c8f14d0c3"
server_key_version: "encryptionkey@0"
I20260812 01:13:37.091285 21779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:37.104651 21779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:37.107213 21779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:37.107402 21779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:37.112053 21779 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:36357
I20260812 01:13:37.112100 21869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:36357 every 8 connection(s)
I20260812 01:13:37.112526 21779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:37.113600 21870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:37.115995 21870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca: Bootstrap starting.
I20260812 01:13:37.116645 21870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:37.116978 21870 log.cc:824] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:37.117312 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21779
I20260812 01:13:37.117443 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:37.117643 17928 external_mini_cluster.cc:1428] Setting key 23d662a6798d57236b9cbdcceeab5860
I20260812 01:13:37.117930 21870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca: No bootstrap required, opened a new log
I20260812 01:13:37.120007 21870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c8ac4c852b94dd89bde4353c64970ca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36357 } }
I20260812 01:13:37.120190 21870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:37.120242 21870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c8ac4c852b94dd89bde4353c64970ca, State: Initialized, Role: FOLLOWER
I20260812 01:13:37.120421 21870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c8ac4c852b94dd89bde4353c64970ca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36357 } }
I20260812 01:13:37.120527 21870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:37.120573 21870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:37.120623 21870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:37.121464 21870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c8ac4c852b94dd89bde4353c64970ca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36357 } }
I20260812 01:13:37.121599 21870 leader_election.cc:302] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c8ac4c852b94dd89bde4353c64970ca; no voters:
I20260812 01:13:37.121809 21870 leader_election.cc:288] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:37.121866 21879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:37.122057 21879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [term 1 LEADER]: Becoming Leader. State: Replica: 9c8ac4c852b94dd89bde4353c64970ca, State: Running, Role: LEADER
I20260812 01:13:37.122119 21870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:37.122203 21879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c8ac4c852b94dd89bde4353c64970ca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36357 } }
I20260812 01:13:37.122552 21879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c8ac4c852b94dd89bde4353c64970ca. Latest consensus state: current_term: 1 leader_uuid: "9c8ac4c852b94dd89bde4353c64970ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c8ac4c852b94dd89bde4353c64970ca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36357 } } }
I20260812 01:13:37.122787 21879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:37.122722 21880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c8ac4c852b94dd89bde4353c64970ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c8ac4c852b94dd89bde4353c64970ca" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 36357 } } }
I20260812 01:13:37.123270 21880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:37.123458 21895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:37.123994 21895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:37.125844 21895 catalog_manager.cc:1347] Generated new cluster ID: ae3cf8fe911f483ea81cfc2dddec904a
I20260812 01:13:37.125900 21895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 01:13:37 dist-test-slave-tfpf krb5kdc[21761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:37.131174 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:37.118334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44167 (local address 127.17.130.62:36357)
0812 01:13:37.118558 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:37.118567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:37.118593 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:13:37.119262 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:37.119270 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:37.119322 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:37.119478 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:37.120150 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:37.121893 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:37.123166 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:37.123858 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:37.123995 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:37.126972 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:37.126976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:37.126977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:37.126989 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:37.129302 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:37.129766 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:37.129769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:37.129770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:37.129878 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:37.130105 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:37.130106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:37.130106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:37.130415 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:37.130663 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:37.130761 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 01:13:37.130957 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260812 01:13:37.132426 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:37.137316 21895 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:37.138015 21895 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:37.143056 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:37.134158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49203 (local address 127.17.130.62:36357)
0812 01:13:37.134201 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:37.134203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:37.134351 (+ 148us) server_negotiation.cc:407] Connection header received
0812 01:13:37.134407 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:37.134408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:37.134420 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:37.134494 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:37.135193 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:37.135958 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:37.137389 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:37.137681 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:37.137766 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:37.139601 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:37.139602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:37.139602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:37.139611 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:37.141677 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:37.142154 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:37.142154 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:37.142155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:37.142225 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:37.142469 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:37.142469 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:37.142470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:37.142595 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:37.142683 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:37.142852 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 01:13:37.142955 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 01:13:37.144076 21895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c8ac4c852b94dd89bde4353c64970ca: Generated new TSK 0
I20260812 01:13:37.144294 21895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:37.147154 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21779
2026-08-12T01:13:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1654 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:37 dist-test-slave-tfpf krb5kdc[21915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:37 dist-test-slave-tfpf krb5kdc[21915](info): set up 2 sockets
Aug 12 01:13:37 dist-test-slave-tfpf krb5kdc[21915](info): commencing operation
krb5kdc: starting...
W20260812 01:13:38.496208 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:38 dist-test-slave-tfpf krb5kdc[21915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:38.641427 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:45321
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:33111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:45321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:38.735450 21941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:38.735749 21941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:38.735793 21941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:38.737008 21941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:38.737110 21941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:38.737149 21941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:38.737174 21941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:38.737201 21941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:38.738773 21941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:45321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:45321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:38.739060 21941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:38.739318 21941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:38.744679 21948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:38.744750 21947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:38.745098 21941 server_base.cc:1034] running on GCE node
W20260812 01:13:38.744845 21951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:38.745411 21941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:38.745749 21941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:38.746940 21941 hybrid_clock.cc:648] HybridClock initialized: now 1786497218746918 us; error 29 us; skew 500 ppm
Aug 12 01:13:38 dist-test-slave-tfpf krb5kdc[21915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497218, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:38.750002 21941 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:38.750689 21941 webserver.cc:460] Webserver started at http://127.17.130.62:44039/ using document root <none> and password file <none>
I20260812 01:13:38.751014 21941 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:38.751112 21941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:38.751276 21941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:38.752506 21941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "151f60b9cfb140a9b8b7562535b4c0ab"
format_stamp: "Formatted at 2026-08-12 01:13:38 on dist-test-slave-tfpf"
server_key: "9db055cb98564e214aed355f4771b238"
server_key_iv: "f474c1d8a084d96c43f0671e676704a3"
server_key_version: "encryptionkey@0"
I20260812 01:13:38.752931 21941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "151f60b9cfb140a9b8b7562535b4c0ab"
format_stamp: "Formatted at 2026-08-12 01:13:38 on dist-test-slave-tfpf"
server_key: "9db055cb98564e214aed355f4771b238"
server_key_iv: "f474c1d8a084d96c43f0671e676704a3"
server_key_version: "encryptionkey@0"
I20260812 01:13:38.754571 21941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:38.755569 21965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:38.755846 21941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:38.755975 21941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "151f60b9cfb140a9b8b7562535b4c0ab"
format_stamp: "Formatted at 2026-08-12 01:13:38 on dist-test-slave-tfpf"
server_key: "9db055cb98564e214aed355f4771b238"
server_key_iv: "f474c1d8a084d96c43f0671e676704a3"
server_key_version: "encryptionkey@0"
I20260812 01:13:38.756062 21941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:38.769752 21941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:38.771996 21941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:38.772193 21941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:38.776669 21941 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:45321
I20260812 01:13:38.776713 22045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:45321 every 8 connection(s)
I20260812 01:13:38.777184 21941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:38.778163 22046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:38.779495 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 21941
I20260812 01:13:38.779626 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:38.779803 17928 external_mini_cluster.cc:1428] Setting key b79a7fe1b27c640b60c71f756d5b9812
I20260812 01:13:38.781387 22046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab: Bootstrap starting.
I20260812 01:13:38.782152 22046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:38.782460 22046 log.cc:824] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:38.783301 22046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab: No bootstrap required, opened a new log
I20260812 01:13:38.785741 22046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45321 } }
I20260812 01:13:38.785888 22046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:38.785912 22046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 151f60b9cfb140a9b8b7562535b4c0ab, State: Initialized, Role: FOLLOWER
I20260812 01:13:38.786046 22046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45321 } }
I20260812 01:13:38.786140 22046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:38.786178 22046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:38.786258 22046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:38.786592 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:38.780467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40507 (local address 127.17.130.62:45321)
0812 01:13:38.780685 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:38.780693 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:38.780721 (+ 28us) server_negotiation.cc:407] Connection header received
0812 01:13:38.781410 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:38.781418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:38.781505 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:38.781642 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:38.781931 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:38.783039 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:38.784157 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:38.784663 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:38.784764 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:38.785608 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:38.785612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:38.785612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:38.785808 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:38.785857 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:38.785869 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:38.786073 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:38.786189 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 01:13:38.786424 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260812 01:13:38.786993 22046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45321 } }
I20260812 01:13:38.787106 22046 leader_election.cc:302] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 151f60b9cfb140a9b8b7562535b4c0ab; no voters:
I20260812 01:13:38.787324 22046 leader_election.cc:288] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:38.787397 22054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:38.787601 22046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:38.787699 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:38.787608 22054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [term 1 LEADER]: Becoming Leader. State: Replica: 151f60b9cfb140a9b8b7562535b4c0ab, State: Running, Role: LEADER
I20260812 01:13:38.787837 22054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45321 } }
I20260812 01:13:38.788214 22054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 151f60b9cfb140a9b8b7562535b4c0ab. Latest consensus state: current_term: 1 leader_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45321 } } }
I20260812 01:13:38.788322 22054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:38.788590 22068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:38.789199 22068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:38.789381 22056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "151f60b9cfb140a9b8b7562535b4c0ab" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 45321 } } }
I20260812 01:13:38.789623 22056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:38.790772 22068 catalog_manager.cc:1347] Generated new cluster ID: 6f1657847a1c489da6abaa6e8334d4db
I20260812 01:13:38.790830 22068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:38.800884 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:38.793561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46709 (local address 127.17.130.62:45321)
0812 01:13:38.793623 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:38.793625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:38.797329 (+ 3704us) server_negotiation.cc:407] Connection header received
0812 01:13:38.797393 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:38.797394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:38.797407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:38.797485 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:38.797649 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:38.798452 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:38.799432 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:38.799762 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:38.799837 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:38.800610 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:38.800612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:38.800612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:38.800619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:38.800637 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:38.800642 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:38.800701 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:38.800762 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 01:13:38.800792 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 01:13:38.804193 22068 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:38.804797 22068 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:38.812124 22068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 151f60b9cfb140a9b8b7562535b4c0ab: Generated new TSK 0
I20260812 01:13:38.812366 22068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:38.822599 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 21941
2026-08-12T01:13:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1663 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:38 dist-test-slave-tfpf krb5kdc[22095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:38 dist-test-slave-tfpf krb5kdc[22095](info): set up 2 sockets
Aug 12 01:13:38 dist-test-slave-tfpf krb5kdc[22095](info): commencing operation
krb5kdc: starting...
W20260812 01:13:40.164258 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:40 dist-test-slave-tfpf krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:40.313647 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33065
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:41015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:33065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:40.404264 22121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:40.404486 22121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:40.404510 22121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:40.405704 22121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:40.405750 22121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:40.405766 22121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:40.405777 22121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:40.405790 22121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:40.407297 22121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:41015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:33065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:33065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:40.407563 22121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:40.407811 22121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:40.413117 22131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:40.413128 22132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:40.413115 22136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:40.413354 22121 server_base.cc:1034] running on GCE node
I20260812 01:13:40.413712 22121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:40.414089 22121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:40.415264 22121 hybrid_clock.cc:648] HybridClock initialized: now 1786497220415238 us; error 47 us; skew 500 ppm
Aug 12 01:13:40 dist-test-slave-tfpf krb5kdc[22095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497220, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:40.417918 22121 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:40.418322 22121 webserver.cc:460] Webserver started at http://127.17.130.62:45231/ using document root <none> and password file <none>
I20260812 01:13:40.418510 22121 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:40.418562 22121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:40.418656 22121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:40.419679 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "cb68186b85ea4e27ae958bc27ca9705b"
format_stamp: "Formatted at 2026-08-12 01:13:40 on dist-test-slave-tfpf"
server_key: "d439b7802afdbcbfabafb00bb4c52f1a"
server_key_iv: "835dc82a3f9e04c5938e6a20ea19dbb6"
server_key_version: "encryptionkey@0"
I20260812 01:13:40.419997 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "cb68186b85ea4e27ae958bc27ca9705b"
format_stamp: "Formatted at 2026-08-12 01:13:40 on dist-test-slave-tfpf"
server_key: "d439b7802afdbcbfabafb00bb4c52f1a"
server_key_iv: "835dc82a3f9e04c5938e6a20ea19dbb6"
server_key_version: "encryptionkey@0"
I20260812 01:13:40.421418 22121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 01:13:40.422336 22144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:40.422559 22121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:40.422665 22121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "cb68186b85ea4e27ae958bc27ca9705b"
format_stamp: "Formatted at 2026-08-12 01:13:40 on dist-test-slave-tfpf"
server_key: "d439b7802afdbcbfabafb00bb4c52f1a"
server_key_iv: "835dc82a3f9e04c5938e6a20ea19dbb6"
server_key_version: "encryptionkey@0"
I20260812 01:13:40.422725 22121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:40.426153 22121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:40.428506 22121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:40.428622 22121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:40.433187 22121 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:33065
I20260812 01:13:40.433216 22222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:33065 every 8 connection(s)
I20260812 01:13:40.433593 22121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:40.434746 22224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:40.437311 22224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b: Bootstrap starting.
I20260812 01:13:40.438086 22224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:40.438434 22224 log.cc:824] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:40.439241 22224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b: No bootstrap required, opened a new log
I20260812 01:13:40.440657 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22121
I20260812 01:13:40.440762 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:40.441180 17928 external_mini_cluster.cc:1428] Setting key fe139daa00d7969581859a219eef0530
I20260812 01:13:40.441151 22224 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb68186b85ea4e27ae958bc27ca9705b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33065 } }
I20260812 01:13:40.441362 22224 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 01:13:40.441417 22224 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb68186b85ea4e27ae958bc27ca9705b, State: Initialized, Role: FOLLOWER
I20260812 01:13:40.441573 22224 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb68186b85ea4e27ae958bc27ca9705b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33065 } }
I20260812 01:13:40.441718 22224 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 01:13:40.441772 22224 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 01:13:40.441867 22224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:40.443182 22224 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb68186b85ea4e27ae958bc27ca9705b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33065 } }
I20260812 01:13:40.443316 22224 leader_election.cc:302] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb68186b85ea4e27ae958bc27ca9705b; no voters:
I20260812 01:13:40.443518 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:40.441761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34699 (local address 127.17.130.62:33065)
0812 01:13:40.441945 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:40.441955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:40.441990 (+ 35us) server_negotiation.cc:407] Connection header received
0812 01:13:40.442112 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:40.442118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:40.442141 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:40.442252 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:40.442965 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:40.442969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:40.442969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:40.442973 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:40.443015 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:40.443027 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:40.443083 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:40.443180 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 01:13:40.443213 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260812 01:13:40.443771 22230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:40.444046 22230 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [term 1 LEADER]: Becoming Leader. State: Replica: cb68186b85ea4e27ae958bc27ca9705b, State: Running, Role: LEADER
I20260812 01:13:40.444164 22230 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb68186b85ea4e27ae958bc27ca9705b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33065 } }
I20260812 01:13:40.443748 22224 leader_election.cc:288] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 01:13:40.444590 22230 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb68186b85ea4e27ae958bc27ca9705b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb68186b85ea4e27ae958bc27ca9705b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33065 } } }
I20260812 01:13:40.444929 22230 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:40.445055 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:40.444725 22232 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb68186b85ea4e27ae958bc27ca9705b. Latest consensus state: current_term: 1 leader_uuid: "cb68186b85ea4e27ae958bc27ca9705b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb68186b85ea4e27ae958bc27ca9705b" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 33065 } } }
I20260812 01:13:40.445153 22232 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:40.444768 22224 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:40.445267 22235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:40.445938 22235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:40.448167 22235 catalog_manager.cc:1347] Generated new cluster ID: 9d69464b2c794f98b2ce3ad57425b0fa
I20260812 01:13:40.448226 22235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:40.448561 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:40.447202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35945 (local address 127.17.130.62:33065)
0812 01:13:40.447236 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:40.447238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:40.447469 (+ 231us) server_negotiation.cc:407] Connection header received
0812 01:13:40.447541 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:40.447541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:40.447546 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:40.447625 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:40.448320 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:40.448321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:40.448321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 01:13:40.448325 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:40.448353 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 01:13:40.448359 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:40.448408 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:40.448478 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 01:13:40.448494 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 01:13:40.456528 22235 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:40.456758 22235 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:40.465543 22235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb68186b85ea4e27ae958bc27ca9705b: Generated new TSK 0
I20260812 01:13:40.465804 22235 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 01:13:40.474025 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22121
2026-08-12T01:13:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1658 ms)
[----------] 18 tests from AuthTokenIssuingTest (30226 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:40 dist-test-slave-tfpf krb5kdc[22279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:40 dist-test-slave-tfpf krb5kdc[22279](info): set up 2 sockets
Aug 12 01:13:40 dist-test-slave-tfpf krb5kdc[22279](info): commencing operation
krb5kdc: starting...
W20260812 01:13:41.889437 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.380s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:41 dist-test-slave-tfpf krb5kdc[22279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:42.038062 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43805
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:43805,127.17.130.61:40441,127.17.130.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:42.132263 22302 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:13:42.132560 22302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:42.132594 22302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:42.132640 22302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:42.133910 22302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:42.133984 22302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:42.134016 22302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:42.134060 22302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:42.134092 22302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:42.135679 22302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:43805,127.17.130.61:40441,127.17.130.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:43805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:42.135993 22302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:42.136276 22302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:42.141520 22307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:42.141518 22311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:42.141584 22308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:42.142232 22302 server_base.cc:1034] running on GCE node
I20260812 01:13:42.142491 22302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:42.142833 22302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:42.144032 22302 hybrid_clock.cc:648] HybridClock initialized: now 1786497222144016 us; error 36 us; skew 500 ppm
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:42.146945 22302 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:42.147375 22302 webserver.cc:460] Webserver started at http://127.17.130.62:43189/ using document root <none> and password file <none>
I20260812 01:13:42.147570 22302 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:42.147611 22302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:42.147716 22302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:42.148761 22302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "623be39cccc6496692fa0f7bcdaf7847"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "d6267f8e501f0daa682aab44415bf483"
server_key_iv: "3ed114a5bbabb54eb9a5501971b67fdb"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.149147 22302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "623be39cccc6496692fa0f7bcdaf7847"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "d6267f8e501f0daa682aab44415bf483"
server_key_iv: "3ed114a5bbabb54eb9a5501971b67fdb"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.150508 22302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:42.151351 22322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:42.151641 22302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:42.151763 22302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "623be39cccc6496692fa0f7bcdaf7847"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "d6267f8e501f0daa682aab44415bf483"
server_key_iv: "3ed114a5bbabb54eb9a5501971b67fdb"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.151850 22302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:42.169524 22302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:42.172057 22302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:42.172251 22302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:42.177045 22302 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:43805
I20260812 01:13:42.177165 22392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:43805 every 8 connection(s)
I20260812 01:13:42.177549 22302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:42.178685 22393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:42.180616 22393 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:42.185042 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22302
I20260812 01:13:42.185195 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:42.185390 17928 external_mini_cluster.cc:1428] Setting key fc0c55a47a3527804200816e6b71dea9
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:42.192631 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.181562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39272 (local address 127.17.130.62:43805)
0812 01:13:42.181790 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.181802 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.181871 (+ 69us) server_negotiation.cc:407] Connection header received
0812 01:13:42.182582 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.182586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.182617 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.182773 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.183465 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.184355 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.185563 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.186069 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.186206 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.188893 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.188904 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.188905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.188914 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.190763 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.191275 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.191279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.191279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.191370 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.191687 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.191688 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.191688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.191956 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.192049 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.192308 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.192431 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
I20260812 01:13:42.192711 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.181472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43805 (local address 127.0.0.1:39272)
0812 01:13:42.181759 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.181797 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.182305 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.182834 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.182838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.183119 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.183319 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.183323 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.184471 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.184473 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.185393 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.185396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.186228 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.186229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.186297 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.186949 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.186959 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.188795 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.190862 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.190870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.190873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.191166 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.191455 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.191456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.191465 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.191589 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.192045 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.192054 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.192140 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.192262 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.192546 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":94,"thread_start_us":157,"threads_started":1}
I20260812 01:13:42.193569 22393 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:42.194007 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.193858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47310)
0812 01:13:42.193917 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.193972 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260812 01:13:42.194185 22326 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
I20260812 01:13:42.194761 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.194652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47324)
0812 01:13:42.194705 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.194737 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 01:13:42.194916 22393 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:40441: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.17.130.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab.
Entry for principal kudu/127.17.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab.
I20260812 01:13:42.216650 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:40441
--webserver_interface=127.17.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab
--principal=kudu/127.17.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:43805,127.17.130.61:40441,127.17.130.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:13:42.244189 22393 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } attempt: 1
I20260812 01:13:42.244905 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.244734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47328)
0812 01:13:42.244810 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.244864 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 01:13:42.245615 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.245407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47344)
0812 01:13:42.245552 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.245589 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260812 01:13:42.245779 22393 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:40441: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
W20260812 01:13:42.310751 22407 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:13:42.311035 22407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:42.311079 22407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:42.311141 22407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:42.312383 22407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:42.312458 22407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:42.312484 22407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:42.312515 22407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:42.312546 22407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:42.313990 22393 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } attempt: 2
I20260812 01:13:42.314409 22407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:43805,127.17.130.61:40441,127.17.130.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab
--principal=kudu/127.17.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:40441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:42.314564 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.314376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47348)
0812 01:13:42.314455 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.314501 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260812 01:13:42.314728 22407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:42.314985 22407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:13:42.315006 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.314898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47358)
0812 01:13:42.314948 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.314979 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:13:42.315147 22393 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:40441: Network error: Client connection negotiation failed: client connection to 127.17.130.61:40441: connect: Connection refused (error 111)
W20260812 01:13:42.321023 22419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:42.321098 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:42.321051 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:42.321553 22407 server_base.cc:1034] running on GCE node
I20260812 01:13:42.321909 22407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:42.322250 22407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:42.323441 22407 hybrid_clock.cc:648] HybridClock initialized: now 1786497222323405 us; error 51 us; skew 500 ppm
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:42.326439 22407 init.cc:377] Logged in from keytab as kudu/127.17.130.61@KRBTEST.COM (short username kudu)
I20260812 01:13:42.326982 22407 webserver.cc:460] Webserver started at http://127.17.130.61:43501/ using document root <none> and password file <none>
I20260812 01:13:42.327243 22407 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:42.327314 22407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:42.327466 22407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:42.328636 22407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data/instance:
uuid: "c8d9a74a8f0c4c4780103b45b51bb03b"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "b39ee8a1ea21c724a880e4b562ceeaf3"
server_key_iv: "234741c277b0f6f2e3c1f0267a8263c0"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.329026 22407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal/instance:
uuid: "c8d9a74a8f0c4c4780103b45b51bb03b"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "b39ee8a1ea21c724a880e4b562ceeaf3"
server_key_iv: "234741c277b0f6f2e3c1f0267a8263c0"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.331028 22407 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260812 01:13:42.331894 22428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:42.332165 22407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 01:13:42.332261 22407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal
uuid: "c8d9a74a8f0c4c4780103b45b51bb03b"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "b39ee8a1ea21c724a880e4b562ceeaf3"
server_key_iv: "234741c277b0f6f2e3c1f0267a8263c0"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.332365 22407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:42.357313 22407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:42.360562 22407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:42.360762 22407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:42.365733 22407 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.61:40441
I20260812 01:13:42.365813 22507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.61:40441 every 8 connection(s)
I20260812 01:13:42.366240 22407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
I20260812 01:13:42.367673 22508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:42.369488 22508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:42.373981 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22407
I20260812 01:13:42.374125 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-1/wal/instance
I20260812 01:13:42.374336 17928 external_mini_cluster.cc:1428] Setting key 99b4c28bc00bed0e82aace9f48e4c0d9
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:42.380478 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.370334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39278 (local address 127.17.130.62:43805)
0812 01:13:42.370436 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.370438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.370578 (+ 140us) server_negotiation.cc:407] Connection header received
0812 01:13:42.371114 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.371115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.371127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.371190 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.371893 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.372397 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.373643 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.373901 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.374103 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.376955 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.376957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.376957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.376967 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.378632 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.379369 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.379371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.379371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.379446 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.379732 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.379734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.379734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.379879 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.379976 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.380279 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.380378 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 01:13:42.380694 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.370242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43805 (local address 127.0.0.1:39278)
0812 01:13:42.370451 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.370492 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.370992 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.371276 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.371279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.371550 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.371769 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.371773 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.372518 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.372519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.373514 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.373517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.374137 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.374139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.374306 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.374919 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.374928 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.376646 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.378754 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.378765 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.378767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.379249 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.379531 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.379532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.379532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.379659 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.379971 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.379979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.380138 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.380257 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.380437 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":86,"threads_started":1}
I20260812 01:13:42.381338 22508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.130.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.60@KRBTEST.COM" created.
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:13:42.391149 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.381688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47374)
0812 01:13:42.381746 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.381758 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.381802 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.382219 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.382220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.382390 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.382446 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.382448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.383413 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.383414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.383963 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.383964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.384548 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.384549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.384623 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.385140 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.385146 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.387852 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.389934 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.389935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.389936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.390178 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.390428 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.390429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.390430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.390511 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.390866 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.390867 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.390909 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.390966 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.391057 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 01:13:42.391331 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.381774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47374 (local address 127.17.130.61:40441)
0812 01:13:42.382029 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.382033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.382056 (+ 23us) server_negotiation.cc:407] Connection header received
0812 01:13:42.382078 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.382080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.382092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.382217 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.382536 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.383298 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.384065 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.384454 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.384552 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.387947 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.387958 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.387958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.387967 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.389842 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.390284 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.390287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.390287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.390350 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.390577 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.390578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.390578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.390795 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.390892 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.391005 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.391174 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":219,"thread_start_us":123,"threads_started":1}
I20260812 01:13:42.391976 22508 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:42.392364 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.392247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52546)
0812 01:13:42.392295 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.392343 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 01:13:42.392544 22429 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
I20260812 01:13:42.393124 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.392941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52558)
0812 01:13:42.393018 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.393049 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 01:13:42.393267 22508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:32811: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab.
Entry for principal kudu/127.17.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab.
I20260812 01:13:42.406296 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:32811
--webserver_interface=127.17.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab
--principal=kudu/127.17.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:43805,127.17.130.61:40441,127.17.130.60:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:13:42.406339 22393 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } attempt: 3
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:13:42.414124 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.406830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.17.130.61:40441)
0812 01:13:42.406900 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.406903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.406933 (+ 30us) server_negotiation.cc:407] Connection header received
0812 01:13:42.406970 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.406970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.406980 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.407043 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.407389 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.407893 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.408608 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.408836 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.408914 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.411177 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.411179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.411179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.411190 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.412823 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.413283 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.413284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.413285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.413349 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.413563 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.413564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.413564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.413694 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.413766 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.413932 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.414020 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:13:42.414124 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.406744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47386)
0812 01:13:42.406809 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.406821 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.406896 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.407049 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.407050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.407230 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.407298 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.407300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.407985 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.407987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.408518 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.408520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.408925 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.408926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.409010 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.409566 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.409572 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.411064 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.412938 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.412939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.412939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.413200 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.413424 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.413425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.413425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.413492 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.413772 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.413773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.413823 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.413905 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.414018 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 01:13:42.414700 22393 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:42.415108 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.414993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52564)
0812 01:13:42.415045 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.415074 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 01:13:42.415637 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.415488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52568)
0812 01:13:42.415553 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.415601 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 01:13:42.415853 22393 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:32811: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
I20260812 01:13:42.442521 22508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } attempt: 1
I20260812 01:13:42.443075 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.442919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52584)
0812 01:13:42.442984 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.443026 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 01:13:42.443439 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.443338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52586)
0812 01:13:42.443384 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.443413 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 01:13:42.443536 22508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:32811: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
I20260812 01:13:42.447031 22393 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } attempt: 1
I20260812 01:13:42.447520 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.447381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52588)
0812 01:13:42.447444 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.447487 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:13:42.447885 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.447783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52604)
0812 01:13:42.447831 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.447863 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 01:13:42.448015 22393 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:32811: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
W20260812 01:13:42.502355 22518 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:13:42.502583 22518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:42.502601 22518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:42.502629 22518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:42.503877 22518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:42.503924 22518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:42.503939 22518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:42.503952 22518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:42.503964 22518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:42.505573 22518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:43805,127.17.130.61:40441,127.17.130.60:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab
--principal=kudu/127.17.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:42.505858 22518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:42.506124 22518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:42.511322 22527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:42.511340 22524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:42.511624 22518 server_base.cc:1034] running on GCE node
W20260812 01:13:42.511399 22525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:42.511703 22508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } attempt: 2
I20260812 01:13:42.511968 22518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:42.512159 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.511993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52614)
0812 01:13:42.512079 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.512118 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 01:13:42.512329 22518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:42.512518 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.512417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52616)
0812 01:13:42.512451 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.512482 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 01:13:42.512633 22508 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:32811: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
I20260812 01:13:42.513499 22518 hybrid_clock.cc:648] HybridClock initialized: now 1786497222513473 us; error 44 us; skew 500 ppm
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:42.516299 22518 init.cc:377] Logged in from keytab as kudu/127.17.130.60@KRBTEST.COM (short username kudu)
I20260812 01:13:42.516810 22518 webserver.cc:460] Webserver started at http://127.17.130.60:39245/ using document root <none> and password file <none>
I20260812 01:13:42.517222 22518 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:42.517326 22518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:42.517482 22518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:42.518821 22518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data/instance:
uuid: "84ff089cd833400e91527095bd1aad0a"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "f8dc06dfce5681eed129054ced5514b9"
server_key_iv: "a01e16c3affc9710044eab03ab70e93d"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.519274 22518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal/instance:
uuid: "84ff089cd833400e91527095bd1aad0a"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "f8dc06dfce5681eed129054ced5514b9"
server_key_iv: "a01e16c3affc9710044eab03ab70e93d"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.520939 22518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260812 01:13:42.521914 22537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:42.522179 22518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:42.522315 22518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal
uuid: "84ff089cd833400e91527095bd1aad0a"
format_stamp: "Formatted at 2026-08-12 01:13:42 on dist-test-slave-tfpf"
server_key: "f8dc06dfce5681eed129054ced5514b9"
server_key_iv: "a01e16c3affc9710044eab03ab70e93d"
server_key_version: "encryptionkey@0"
I20260812 01:13:42.522404 22518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:42.523178 22393 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } attempt: 2
I20260812 01:13:42.523656 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.523511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52620)
0812 01:13:42.523585 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.523625 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 01:13:42.524150 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.524009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52632)
0812 01:13:42.524095 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.524127 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260812 01:13:42.524283 22393 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:32811: Network error: Client connection negotiation failed: client connection to 127.17.130.60:32811: connect: Connection refused (error 111)
I20260812 01:13:42.542685 22518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:42.546149 22518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:42.546339 22518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:42.551829 22518 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.60:32811
I20260812 01:13:42.552035 22618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.60:32811 every 8 connection(s)
I20260812 01:13:42.552386 22518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
I20260812 01:13:42.553906 22621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:42.554399 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22518
I20260812 01:13:42.554522 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786496918848844-17928-0/minicluster-data/master-2/wal/instance
I20260812 01:13:42.554750 17928 external_mini_cluster.cc:1428] Setting key d2f62cf5e47cabc4fb032f66c77f3e93
I20260812 01:13:42.557482 22621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:42.568571 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.558395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.17.130.62:43805)
0812 01:13:42.558479 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.558482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.558734 (+ 252us) server_negotiation.cc:407] Connection header received
0812 01:13:42.558767 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.558768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.558781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.558862 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.559369 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.560285 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.561246 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.561508 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.561578 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.564502 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.564503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.564504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.564515 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.566834 (+ 2319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.567390 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.567391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.567392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.567473 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.567791 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.567792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.567792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.567942 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.568026 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.568357 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.568460 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 01:13:42.568837 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.558348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:43805 (local address 127.0.0.1:39286)
0812 01:13:42.558571 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.558617 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.558701 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.558908 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.558912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.559204 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.559278 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.559280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.560376 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.560377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.561157 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.561160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.561652 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.561653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.561753 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.562372 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.562383 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.564346 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.567000 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.567012 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.567015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.567279 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.567555 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.567556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.567557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.567699 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.568031 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.568040 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.568203 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.568331 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.568514 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":132,"threads_started":1}
I20260812 01:13:42.568840 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.555472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52648 (local address 127.17.130.60:32811)
0812 01:13:42.555660 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.555672 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.555714 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:13:42.556251 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.556258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.556311 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.556432 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.557458 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.558615 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.559998 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.560523 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.560667 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.564115 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.564131 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.564131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.564142 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.566811 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.567669 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.567674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.567731 (+ 57us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.567820 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.568140 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.568141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.568141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.568298 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.568455 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.568569 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.568659 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260812 01:13:42.569566 22621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:42.578528 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.569851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:40441 (local address 127.0.0.1:47394)
0812 01:13:42.569889 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.569902 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.569948 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.570190 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.570191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.570365 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.570416 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.570419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.571329 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.571331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.572226 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.572229 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.572736 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.572737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.572792 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.573338 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.573343 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.574985 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.577266 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.577267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.577268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.577461 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.577732 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.577733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.577733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.577800 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.578139 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.578140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.578196 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.578265 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.578381 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 01:13:42.578572 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.569995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.17.130.61:40441)
0812 01:13:42.570066 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.570069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.570079 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:42.570109 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.570110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.570122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.570180 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.570521 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.571205 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.572354 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.572647 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.572729 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.575095 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.575097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.575097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.575107 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.577153 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.577553 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.577555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.577555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.577627 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.577929 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.577930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.577930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.578062 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.578121 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.578346 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.578440 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 01:13:42.578995 22621 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:42.579502 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.570146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.17.130.62:43805)
0812 01:13:42.570185 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.570187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.570194 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:13:42.570217 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.570218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.570226 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.570281 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.570845 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.571455 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.572536 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.572828 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.572888 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.575804 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.575806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.575806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.575815 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.577793 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.578203 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.578204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.578204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.578284 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.578511 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.578511 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.578512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.578677 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.578855 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.579304 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.579392 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:13:42.588264 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.579372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52658)
0812 01:13:42.579425 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.579437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.579482 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.579674 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.579675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.579821 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.579862 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.579863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.580798 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.580799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.581426 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.581429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.582080 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.582081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.582172 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.582692 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.582698 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.584430 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.586479 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.586480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.586481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.586661 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.587057 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.587059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.587059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.587246 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.587896 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.587898 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.587948 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.588045 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.588166 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:13:42.588475 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.579419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.17.130.60:32811)
0812 01:13:42.579543 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.579545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.579552 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:13:42.579572 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.579572 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.579582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.579636 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.579926 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.580711 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.581506 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.581885 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.582048 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.584534 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.584536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.584536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.584545 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.586368 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.586797 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.586799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.586800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.586867 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.587350 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.587351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.587351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.587643 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.587752 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.588138 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.588237 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 01:13:42.590268 22621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: Bootstrap starting.
I20260812 01:13:42.591377 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.580307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.17.130.61:40441)
0812 01:13:42.580347 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.580349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.580358 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:42.580435 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.580436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.580446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.580503 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.581129 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.581720 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.583083 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.583569 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.583661 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.587468 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.587469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.587470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.587477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.589544 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.590004 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.590005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.590006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.590063 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.590321 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.590323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.590323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.590552 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.590875 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.591176 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.591268 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:42.592079 22621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:42.592417 22621 log.cc:824] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:42.593426 22621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: No bootstrap required, opened a new log
I20260812 01:13:42.593636 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:42.595510 22621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.595693 22621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84ff089cd833400e91527095bd1aad0a, State: Initialized, Role: FOLLOWER
I20260812 01:13:42.595788 22621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.594806 22358 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "623be39cccc6496692fa0f7bcdaf7847"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39288
I20260812 01:13:42.596117 22630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.596174 22621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:42.596242 22630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:13:42.598767 22648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:42.598850 22648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:42.599207 22518 master_runner.cc:186] Error getting master registration for 127.17.130.62:43805: OK, instance_id { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" instance_seqno: 1786497222175741 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:13:42.602632 22518 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:13:42.603766 22508 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } attempt: 3
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:13:42.613204 22510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.604175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52672)
0812 01:13:42.604235 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.604250 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.604318 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.604498 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.604499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.604725 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.604792 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.604794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.605795 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.605796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.606605 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.606608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.607110 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.607111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.607172 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.607743 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.607749 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.609852 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.611908 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.611909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.611910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.612111 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.612352 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.612354 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.612354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.612457 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.612809 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.612810 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.612866 (+ 56us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.612946 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.613092 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 01:13:42.613251 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.604253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.17.130.60:32811)
0812 01:13:42.604306 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.604308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.604320 (+ 12us) server_negotiation.cc:407] Connection header received
0812 01:13:42.604387 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.604389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.604399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.604462 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.604883 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.605701 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.606712 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.607016 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.607096 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.609977 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.609979 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.609979 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.609995 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.611822 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.612195 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.612197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.612197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.612260 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.612534 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.612536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.612536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.612732 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.612803 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.613006 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.613146 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:13:42.615089 22508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: Bootstrap starting.
I20260812 01:13:42.615948 22508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:42.616278 22508 log.cc:824] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:42.617368 22508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: No bootstrap required, opened a new log
I20260812 01:13:42.619277 22508 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.619474 22508 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d9a74a8f0c4c4780103b45b51bb03b, State: Initialized, Role: FOLLOWER
I20260812 01:13:42.619604 22508 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.620165 22508 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:42.620146 22651 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.620505 22651 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: This master's current role is: FOLLOWER
W20260812 01:13:42.622634 22670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:42.622797 22670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:42.623406 22393 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } attempt: 3
I20260812 01:13:42.622735 22407 master_runner.cc:186] Error getting master registration for 127.17.130.62:43805: OK, instance_id { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" instance_seqno: 1786497222175741 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:13:42.626833 22407 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 01:13:42 dist-test-slave-tfpf krb5kdc[22279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497222, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:13:42.633566 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.623743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:32811 (local address 127.0.0.1:52682)
0812 01:13:42.623806 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:42.623820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:42.623871 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:42.624146 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:42.624147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:42.624380 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:42.624440 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.624442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.625485 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.625487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.626380 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:42.626383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.626908 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.626909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:42.626993 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.627812 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:42.627819 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:42.630000 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:42.632124 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.632126 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.632126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.632395 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.632628 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:42.632629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:42.632629 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:42.632703 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:42.633097 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:42.633099 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:42.633168 (+ 69us) client_negotiation.cc:769] Sending connection context
0812 01:13:42.633242 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 01:13:42.633361 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 01:13:42.634207 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.623823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52682 (local address 127.17.130.60:32811)
0812 01:13:42.623974 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.623976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.623987 (+ 11us) server_negotiation.cc:407] Connection header received
0812 01:13:42.624017 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.624018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.624030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.624090 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.624535 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.625359 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.626505 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.626826 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.626906 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.630129 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.630131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.630131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.630141 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.631989 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.632490 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.632492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.632492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.632555 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.632774 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.632775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.632775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.632951 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.633018 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.633852 (+ 834us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.634114 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 01:13:42.636327 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847: Bootstrap starting.
I20260812 01:13:42.637389 22393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:42.637763 22393 log.cc:824] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:42.638990 22393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847: No bootstrap required, opened a new log
I20260812 01:13:42.640847 22393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.641047 22393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 623be39cccc6496692fa0f7bcdaf7847, State: Initialized, Role: FOLLOWER
I20260812 01:13:42.641201 22393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.641661 22358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 01:13:42.641624 22677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.641819 22358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:42.641932 22677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:42.642932 22358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.643275 22358 leader_election.cc:288] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [CANDIDATE]: Term 1 election: Requested vote from peers c8d9a74a8f0c4c4780103b45b51bb03b (127.17.130.61:40441), 84ff089cd833400e91527095bd1aad0a (127.17.130.60:32811)
I20260812 01:13:42.641649 22393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:42.643785 22471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "623be39cccc6496692fa0f7bcdaf7847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b"
I20260812 01:13:42.643926 22471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:42.644934 22471 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 623be39cccc6496692fa0f7bcdaf7847 in term 1.
I20260812 01:13:42.646508 22326 leader_election.cc:302] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 623be39cccc6496692fa0f7bcdaf7847, c8d9a74a8f0c4c4780103b45b51bb03b; no voters:
I20260812 01:13:42.646689 22584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "623be39cccc6496692fa0f7bcdaf7847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84ff089cd833400e91527095bd1aad0a"
I20260812 01:13:42.646816 22584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:42.647764 22584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 623be39cccc6496692fa0f7bcdaf7847 in term 1.
I20260812 01:13:42.648345 22677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:42.648991 22677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [term 1 LEADER]: Becoming Leader. State: Replica: 623be39cccc6496692fa0f7bcdaf7847, State: Running, Role: LEADER
I20260812 01:13:42.649216 22677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } }
I20260812 01:13:42.649808 22677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 623be39cccc6496692fa0f7bcdaf7847. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.650219 22677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:42.650255 22302 master_runner.cc:263] No leader master found from master 623be39cccc6496692fa0f7bcdaf7847
I20260812 01:13:42.650347 22302 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:13:42.650568 22701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:42.651268 22701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:42.656301 22584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [term 1 FOLLOWER]: Refusing update from remote peer 623be39cccc6496692fa0f7bcdaf7847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:42.656791 22699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:42.658867 22630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 623be39cccc6496692fa0f7bcdaf7847. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.658984 22630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:42.659085 22471 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [term 1 FOLLOWER]: Refusing update from remote peer 623be39cccc6496692fa0f7bcdaf7847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:42.659456 22699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:42.660949 22651 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 623be39cccc6496692fa0f7bcdaf7847. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.661091 22651 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:42.667296 22699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.667490 22699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:42.668130 22630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.668371 22630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:42.668335 22712 mvcc.cc:205] Tried to move back new op lower bound from 7317492623994507264 to 7317492623972462592. Current Snapshot: MvccSnapshot[applied={T|T < 7317492623994507264}]
I20260812 01:13:42.673914 22677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.674089 22677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:42.674762 22651 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "623be39cccc6496692fa0f7bcdaf7847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "623be39cccc6496692fa0f7bcdaf7847" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 43805 } } peers { permanent_uuid: "c8d9a74a8f0c4c4780103b45b51bb03b" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 40441 } } peers { permanent_uuid: "84ff089cd833400e91527095bd1aad0a" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 32811 } } }
I20260812 01:13:42.674903 22651 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:42.675057 22716 mvcc.cc:205] Tried to move back new op lower bound from 7317492623994507264 to 7317492623972462592. Current Snapshot: MvccSnapshot[applied={T|T < 7317492623994507264}]
I20260812 01:13:42.675789 22701 catalog_manager.cc:1347] Generated new cluster ID: 7330e92a9b1f47c398fc3c6b62de5dbb
I20260812 01:13:42.675864 22701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:42.682664 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.651145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47412 (local address 127.17.130.61:40441)
0812 01:13:42.651189 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.651190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.654350 (+ 3160us) server_negotiation.cc:407] Connection header received
0812 01:13:42.654448 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.654449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.654460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.654491 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.658560 (+ 4069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.659444 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.666890 (+ 7446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.667244 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.667295 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.673916 (+ 6621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.673918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.673919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.673930 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.676276 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.676889 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.676891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.676891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.676961 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.680899 (+ 3938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.680903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.680903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.681312 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.681438 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.681692 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.682537 (+ 845us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:42.688396 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.650757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39294 (local address 127.17.130.62:43805)
0812 01:13:42.650801 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.650803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.653280 (+ 2477us) server_negotiation.cc:407] Connection header received
0812 01:13:42.653390 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.653391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.653402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.653433 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.655770 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.656542 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.665545 (+ 9003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.666050 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.666159 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.677272 (+ 11113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.677273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.677274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.677284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.680215 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.686878 (+ 6663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.686880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.686881 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.686957 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.687173 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.687174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.687174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.687326 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.687413 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.688172 (+ 759us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.688275 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:13:42.690167 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:42.651510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.17.130.60:32811)
0812 01:13:42.651549 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.651551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.654957 (+ 3406us) server_negotiation.cc:407] Connection header received
0812 01:13:42.655529 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.655530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.655542 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.655572 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.660815 (+ 5243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.661560 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.671480 (+ 9920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:42.671855 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:42.671914 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:42.684354 (+ 12440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:42.684356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:42.684357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:42.684367 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:42.686315 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.687801 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.687802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.687803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.687880 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:42.689530 (+ 1650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:42.689532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:42.689532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:42.689691 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:42.689775 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:42.689971 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 01:13:42.690071 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:13:42.705379 22701 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:42.706859 22701 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:42.713155 22701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 623be39cccc6496692fa0f7bcdaf7847: Generated new TSK 0
I20260812 01:13:42.713457 22701 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:13:42.741353 22398 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.739165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39304 (local address 127.17.130.62:43805)
0812 01:13:42.739239 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.739241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.740453 (+ 1212us) server_negotiation.cc:407] Connection header received
0812 01:13:42.740598 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.740599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.740613 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.740654 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.741263 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39304: BlockingRecv error: recv got EOF from 127.0.0.1:39304 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 01:13:42.742338 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.739647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.17.130.61:40441)
0812 01:13:42.739698 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.739700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.741686 (+ 1986us) server_negotiation.cc:407] Connection header received
0812 01:13:42.741827 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.741829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.741842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.741881 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.742260 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47426: BlockingRecv error: recv got EOF from 127.0.0.1:47426 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 01:13:42.743228 22626 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.740086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.17.130.60:32811)
0812 01:13:42.740135 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.740137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.742627 (+ 2490us) server_negotiation.cc:407] Connection header received
0812 01:13:42.742747 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.742748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.742762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.742799 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.743159 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52690: BlockingRecv error: recv got EOF from 127.0.0.1:52690 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 01:13:42.746279 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.745296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.17.130.61:40441)
0812 01:13:42.745339 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.745341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.745629 (+ 288us) server_negotiation.cc:407] Connection header received
0812 01:13:42.745777 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.745778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.745798 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.745835 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.746217 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47436: BlockingRecv error: recv got EOF from 127.0.0.1:47436 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:13:42.749868 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.748961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.17.130.61:40441)
0812 01:13:42.749006 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.749009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.749326 (+ 317us) server_negotiation.cc:407] Connection header received
0812 01:13:42.749425 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.749426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.749437 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.749465 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.749819 (+ 354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47440: BlockingRecv error: recv got EOF from 127.0.0.1:47440 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:13:42.754156 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.753017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47450 (local address 127.17.130.61:40441)
0812 01:13:42.753055 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.753084 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.753374 (+ 290us) server_negotiation.cc:407] Connection header received
0812 01:13:42.753633 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.753635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.753648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.753681 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.754106 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47450: BlockingRecv error: recv got EOF from 127.0.0.1:47450 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 01:13:42.760636 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.759529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47456 (local address 127.17.130.61:40441)
0812 01:13:42.759581 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.759583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.759918 (+ 335us) server_negotiation.cc:407] Connection header received
0812 01:13:42.760055 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.760056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.760070 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.760108 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.760573 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47456: BlockingRecv error: recv got EOF from 127.0.0.1:47456 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 01:13:42.771512 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.770311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47466 (local address 127.17.130.61:40441)
0812 01:13:42.770366 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.770368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.770734 (+ 366us) server_negotiation.cc:407] Connection header received
0812 01:13:42.770876 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.770877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.770893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.770934 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.771443 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47466: BlockingRecv error: recv got EOF from 127.0.0.1:47466 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 01:13:42.790410 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.789196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.17.130.61:40441)
0812 01:13:42.789257 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.789259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.789647 (+ 388us) server_negotiation.cc:407] Connection header received
0812 01:13:42.789787 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.789789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.789803 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.789844 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.790340 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47474: BlockingRecv error: recv got EOF from 127.0.0.1:47474 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 01:13:42.825457 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.824145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47490 (local address 127.17.130.61:40441)
0812 01:13:42.824201 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.824203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.824573 (+ 370us) server_negotiation.cc:407] Connection header received
0812 01:13:42.824713 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.824715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.824731 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.824773 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.825327 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47490: BlockingRecv error: recv got EOF from 127.0.0.1:47490 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260812 01:13:42.891815 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:42.891095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47494 (local address 127.17.130.61:40441)
0812 01:13:42.891213 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:42.891215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:42.891247 (+ 32us) server_negotiation.cc:407] Connection header received
0812 01:13:42.891310 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:42.891311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:42.891322 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:42.891393 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:42.891736 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47494: BlockingRecv error: recv got EOF from 127.0.0.1:47494 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 01:13:43.022732 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:43.021991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.17.130.61:40441)
0812 01:13:43.022080 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.022083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.022175 (+ 92us) server_negotiation.cc:407] Connection header received
0812 01:13:43.022249 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.022250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.022262 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.022308 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:43.022656 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47508: BlockingRecv error: recv got EOF from 127.0.0.1:47508 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 01:13:43.280640 22513 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:43.280050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47524 (local address 127.17.130.61:40441)
0812 01:13:43.280122 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.280128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.280136 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:13:43.280159 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.280159 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.280171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.280224 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:43.280589 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47524: BlockingRecv error: recv got EOF from 127.0.0.1:47524 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 01:13:43.599373 22648 catalog_manager.cc:1259] Loaded cluster ID: 7330e92a9b1f47c398fc3c6b62de5dbb
I20260812 01:13:43.599493 22648 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: loading cluster ID for follower catalog manager: success
I20260812 01:13:43.600607 22648 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: acquiring CA information for follower catalog manager: success
I20260812 01:13:43.601824 22648 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84ff089cd833400e91527095bd1aad0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:13:43.623574 22670 catalog_manager.cc:1259] Loaded cluster ID: 7330e92a9b1f47c398fc3c6b62de5dbb
I20260812 01:13:43.623687 22670 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: loading cluster ID for follower catalog manager: success
I20260812 01:13:43.624531 22670 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: acquiring CA information for follower catalog manager: success
I20260812 01:13:43.624851 22670 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c8d9a74a8f0c4c4780103b45b51bb03b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:13:43.797163 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:43.794059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47534 (local address 127.17.130.61:40441)
0812 01:13:43.794185 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.794187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.794197 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:43.794258 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.794259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.794313 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.794391 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:43.794618 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.795121 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.795957 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.796205 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.796281 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:43.796601 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:43.796747 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:43.796825 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:43.796983 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 01:13:43.797008 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260812 01:13:43.802477 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:43.799641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.17.130.60:32811)
0812 01:13:43.799744 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.799746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.799790 (+ 44us) server_negotiation.cc:407] Connection header received
0812 01:13:43.799875 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.799876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.799912 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.799979 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:43.800196 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.800681 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.801534 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.801728 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.801788 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:43.801996 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:43.802110 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:43.802197 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:43.802276 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 01:13:43.802298 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":51,"threads_started":1}
I20260812 01:13:43.808010 22811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:43.804991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47540 (local address 127.17.130.61:40441)
0812 01:13:43.805081 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.805083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.805288 (+ 205us) server_negotiation.cc:407] Connection header received
0812 01:13:43.805385 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.805386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.805397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.805451 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:43.805599 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.806315 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.807168 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.807428 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.807494 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:43.807730 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:43.807758 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:43.807828 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:43.807893 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 01:13:43.807915 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 01:13:43.808357 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:43.804501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.17.130.60:32811)
0812 01:13:43.804604 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.804606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.804715 (+ 109us) server_negotiation.cc:407] Connection header received
0812 01:13:43.804819 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.804820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.804832 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.804885 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:43.804987 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.805683 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.806718 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.807118 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.807374 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:43.807740 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:43.807775 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:43.808037 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:43.808262 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 01:13:43.808287 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 01:13:43.809052 22828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:43.804500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.17.130.62:43805)
0812 01:13:43.804675 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:43.804677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:43.804962 (+ 285us) server_negotiation.cc:407] Connection header received
0812 01:13:43.805033 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:43.805034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:43.805133 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:43.805244 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:43.805511 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.806545 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.808105 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:43.808389 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:43.808450 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:43.808679 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:43.808783 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:43.808881 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:43.808941 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 01:13:43.808966 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":54,"threads_started":1}
I20260812 01:13:43.810905 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22302
I20260812 01:13:43.828085 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22407
I20260812 01:13:43.845847 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22518
2026-08-12T01:13:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3368 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 01:13:43 dist-test-slave-tfpf krb5kdc[22844](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 01:13:43 dist-test-slave-tfpf krb5kdc[22844](info): set up 2 sockets
Aug 12 01:13:43 dist-test-slave-tfpf krb5kdc[22844](info): commencing operation
krb5kdc: starting...
W20260812 01:13:45.169584 17928 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/test-user.keytab.
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T01:13:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T01:13:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.17.130.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
Entry for principal kudu/127.17.130.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab.
I20260812 01:13:45.309115 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44043
--webserver_interface=127.17.130.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44043,127.17.130.61:36873,127.17.130.60:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 01:13:45.400691 22860 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:13:45.400933 22860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:45.400954 22860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:45.400974 22860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:45.402197 22860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:45.402244 22860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:45.402259 22860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:45.402272 22860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:45.402285 22860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:45.403839 22860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44043,127.17.130.61:36873,127.17.130.60:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.62.keytab
--principal=kudu/127.17.130.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.62:44043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.17.130.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:45.404064 22860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:45.404261 22860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:45.409448 22873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:45.409478 22868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:45.409448 22867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:45.409795 22860 server_base.cc:1034] running on GCE node
I20260812 01:13:45.410077 22860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:45.410343 22860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:45.411543 22860 hybrid_clock.cc:648] HybridClock initialized: now 1786497225411529 us; error 34 us; skew 500 ppm
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:45.414481 22860 init.cc:377] Logged in from keytab as kudu/127.17.130.62@KRBTEST.COM (short username kudu)
I20260812 01:13:45.415002 22860 webserver.cc:460] Webserver started at http://127.17.130.62:36141/ using document root <none> and password file <none>
I20260812 01:13:45.415369 22860 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:45.415429 22860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:45.415583 22860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:45.416749 22860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data/instance:
uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "2e6b22a4e3d758ff4922fced1d173f69"
server_key_iv: "229f2e6594a870f446b4bafdeb18212e"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.417199 22860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal/instance:
uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "2e6b22a4e3d758ff4922fced1d173f69"
server_key_iv: "229f2e6594a870f446b4bafdeb18212e"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.418705 22860 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:45.419723 22886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:45.419945 22860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:45.420047 22860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal
uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "2e6b22a4e3d758ff4922fced1d173f69"
server_key_iv: "229f2e6594a870f446b4bafdeb18212e"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.420115 22860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:45.427466 22860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:45.430455 22860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:45.430588 22860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:45.435087 22860 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.62:44043
I20260812 01:13:45.435123 22956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.62:44043 every 8 connection(s)
I20260812 01:13:45.435532 22860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/data/info.pb
I20260812 01:13:45.436093 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22860
I20260812 01:13:45.436220 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-0/wal/instance
I20260812 01:13:45.436426 17928 external_mini_cluster.cc:1428] Setting key 0441088ec9fd72d56308d6c7373d1543
I20260812 01:13:45.436722 22960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:45.438673 22960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.130.61@KRBTEST.COM; defaulting to no policy
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.61@KRBTEST.COM" created.
I20260812 01:13:45.451370 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.439641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.17.130.62:44043)
0812 01:13:45.439818 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.439828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.439960 (+ 132us) server_negotiation.cc:407] Connection header received
0812 01:13:45.440552 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.440554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.440579 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.440679 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.441322 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.442173 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.443713 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.444060 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.444157 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.447458 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.447466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.447466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.447474 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.449455 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.449896 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.449899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.449899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.449983 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.450329 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.450330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.450330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.450590 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.450697 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.450898 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.451098 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260812 01:13:45.451370 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.439545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44043 (local address 127.0.0.1:37498)
0812 01:13:45.439807 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.439853 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.440385 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.440686 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.440688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.440986 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.441213 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.441218 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.442299 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.442300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.443571 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.443574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.444183 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.444185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.444277 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.444861 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.444872 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.447340 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.449574 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.449582 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.449584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.449816 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.450124 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.450125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.450126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.450239 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.450674 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.450680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.450752 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.450857 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.451148 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":111,"thread_start_us":114,"threads_started":1}
I20260812 01:13:45.452286 22960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:45.452713 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.452567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59656)
0812 01:13:45.452630 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.452688 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 01:13:45.452877 22892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
I20260812 01:13:45.453365 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.453298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59658)
0812 01:13:45.453324 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.453348 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 01:13:45.453490 22960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:36873: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab.
Entry for principal kudu/127.17.130.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab.
I20260812 01:13:45.469868 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:36873
--webserver_interface=127.17.130.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab
--principal=kudu/127.17.130.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44043,127.17.130.61:36873,127.17.130.60:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:13:45.502772 22960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } attempt: 1
I20260812 01:13:45.503455 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.503269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59672)
0812 01:13:45.503350 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.503406 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 01:13:45.504072 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.503961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59680)
0812 01:13:45.504015 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.504047 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 01:13:45.504230 22960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:36873: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
W20260812 01:13:45.565709 22976 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:13:45.565996 22976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:45.566016 22976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:45.566039 22976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:45.567224 22976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:45.567279 22976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:45.567296 22976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:45.567308 22976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:45.567320 22976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:45.568914 22976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44043,127.17.130.61:36873,127.17.130.60:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.61.keytab
--principal=kudu/127.17.130.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.61:36873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.17.130.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:45.569222 22976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:45.569446 22976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 01:13:45.572423 22960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } attempt: 2
I20260812 01:13:45.572968 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.572814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59694)
0812 01:13:45.572887 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.572934 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 01:13:45.573349 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.573241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59698)
0812 01:13:45.573290 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.573318 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260812 01:13:45.573460 22960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.61:36873: Network error: Client connection negotiation failed: client connection to 127.17.130.61:36873: connect: Connection refused (error 111)
I20260812 01:13:45.576068 22976 server_base.cc:1034] running on GCE node
W20260812 01:13:45.576050 22986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:45.576018 22983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:45.576020 22982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:45.576586 22976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:45.576970 22976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:45.578191 22976 hybrid_clock.cc:648] HybridClock initialized: now 1786497225578162 us; error 45 us; skew 500 ppm
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:45.586205 22976 init.cc:377] Logged in from keytab as kudu/127.17.130.61@KRBTEST.COM (short username kudu)
I20260812 01:13:45.586874 22976 webserver.cc:460] Webserver started at http://127.17.130.61:33243/ using document root <none> and password file <none>
I20260812 01:13:45.587365 22976 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:45.587466 22976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:45.587642 22976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:45.588802 22976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data/instance:
uuid: "688a85747fbe4e3793ac67d285f371ff"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "2ba4bd4b94ca09216d81acf1d3802ccf"
server_key_iv: "845fb1456c747ee11089d5f2825ea33d"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.589260 22976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal/instance:
uuid: "688a85747fbe4e3793ac67d285f371ff"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "2ba4bd4b94ca09216d81acf1d3802ccf"
server_key_iv: "845fb1456c747ee11089d5f2825ea33d"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.590847 22976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:45.591768 22997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:45.592089 22976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 01:13:45.592221 22976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal
uuid: "688a85747fbe4e3793ac67d285f371ff"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "2ba4bd4b94ca09216d81acf1d3802ccf"
server_key_iv: "845fb1456c747ee11089d5f2825ea33d"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.592308 22976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:45.601725 22976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:45.604527 22976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:45.604815 22976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:45.609686 22976 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.61:36873
I20260812 01:13:45.609735 23074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.61:36873 every 8 connection(s)
I20260812 01:13:45.610126 22976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/data/info.pb
I20260812 01:13:45.611260 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:45.612931 23076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:45.618068 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 22976
I20260812 01:13:45.618216 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-1/wal/instance
I20260812 01:13:45.618465 17928 external_mini_cluster.cc:1428] Setting key 018e9761bee0230b47ab86dbf9aa06e5
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:45.626138 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.613822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.17.130.62:44043)
0812 01:13:45.613884 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.613886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.614180 (+ 294us) server_negotiation.cc:407] Connection header received
0812 01:13:45.614685 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.614687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.614702 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.614773 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.615525 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.616086 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.617212 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.617462 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.617542 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.621466 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.621468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.621468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.621479 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.623765 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.624486 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.624488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.624489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.624565 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.625121 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.625123 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.625123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.625270 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.625339 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.625891 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.626042 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 01:13:45.626330 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.613765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44043 (local address 127.0.0.1:37502)
0812 01:13:45.613986 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.614035 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.614557 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.614881 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.614885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.615175 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.615382 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.615385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.616205 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.616206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.617055 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.617110 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.617672 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.617675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.617889 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.618806 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.618817 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.621312 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.623901 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.623913 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.623917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.624352 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.624672 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.624673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.624673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.624861 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.625359 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.625368 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.625590 (+ 222us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.625738 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.625964 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":133,"threads_started":1}
I20260812 01:13:45.627261 23076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.17.130.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.17.130.60@KRBTEST.COM" created.
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:13:45.638244 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.627652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.17.130.61:36873)
0812 01:13:45.627825 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.627839 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.627854 (+ 15us) server_negotiation.cc:407] Connection header received
0812 01:13:45.627875 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.627877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.628073 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.628218 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.628675 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.629656 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.630530 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.630888 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.631037 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.634308 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.634329 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.634330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.634337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.636648 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.637165 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.637169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.637169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.637243 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.637505 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.637506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.637506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.637696 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.637788 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.638043 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.638137 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
I20260812 01:13:45.638247 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.627594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59702)
0812 01:13:45.627677 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.627690 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.627742 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.628251 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.628253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.628415 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.628502 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.628504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.629830 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.629831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.630418 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.630420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.631109 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.631111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.631299 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.631993 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.632000 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.634148 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.636781 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.636783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.636783 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.637030 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.637326 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.637328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.637328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.637407 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.637786 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.637787 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.637845 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.637926 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.638051 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 01:13:45.639642 23076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:45.640118 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.639937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46780)
0812 01:13:45.640001 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.640069 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 01:13:45.640309 23001 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
I20260812 01:13:45.641193 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.640965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46794)
0812 01:13:45.641110 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.641153 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260812 01:13:45.641422 23076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:34935: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.17.130.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab.
Entry for principal kudu/127.17.130.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab.
I20260812 01:13:45.653155 17928 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskolrujf/build/release/bin/kudu
/tmp/dist-test-taskolrujf/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:34935
--webserver_interface=127.17.130.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab
--principal=kudu/127.17.130.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.17.130.20:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.17.130.62:44043,127.17.130.61:36873,127.17.130.60:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 01:13:45.664667 22960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } attempt: 3
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:13:45.674335 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.665330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59712 (local address 127.17.130.61:36873)
0812 01:13:45.665406 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.665408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.665421 (+ 13us) server_negotiation.cc:407] Connection header received
0812 01:13:45.665510 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.665511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.665525 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.665788 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.666326 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.666941 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.667827 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.668116 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.668210 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.670868 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.670871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.670871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.670880 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.672637 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.673405 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.673406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.673406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.673468 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.673726 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.673728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.673728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.673891 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.673976 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.674177 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.674264 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 01:13:45.674350 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.665249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59712)
0812 01:13:45.665315 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.665336 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.665418 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.665831 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.665833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.666095 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.666208 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.666210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.667060 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.667062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.667702 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.667704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.668222 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.668224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.668479 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.669130 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.669137 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.670719 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.672764 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.672765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.672766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.673309 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.673538 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.673539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.673539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.673622 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.673973 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.673974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.674038 (+ 64us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.674114 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.674237 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 01:13:45.674831 22960 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:45.675177 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.675083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46808)
0812 01:13:45.675120 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.675153 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 01:13:45.675486 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.675431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46822)
0812 01:13:45.675452 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.675472 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260812 01:13:45.675590 22960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:34935: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
I20260812 01:13:45.690707 23076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } attempt: 1
I20260812 01:13:45.691304 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.691151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46830)
0812 01:13:45.691219 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.691268 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 01:13:45.691810 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.691713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46846)
0812 01:13:45.691761 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.691791 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 01:13:45.692001 23076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:34935: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
I20260812 01:13:45.706770 22960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } attempt: 1
I20260812 01:13:45.707418 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.707254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46858)
0812 01:13:45.707339 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.707385 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 01:13:45.707896 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.707753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46870)
0812 01:13:45.707830 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.707871 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260812 01:13:45.708148 22960 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:34935: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
W20260812 01:13:45.748546 23090 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 01:13:45.748795 23090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 01:13:45.748812 23090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 01:13:45.748835 23090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 01:13:45.750077 23090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 01:13:45.750125 23090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 01:13:45.750140 23090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 01:13:45.750161 23090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 01:13:45.750173 23090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 01:13:45.751713 23090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.17.130.20:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.17.130.62:44043,127.17.130.61:36873,127.17.130.60:34935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/krb5kdc/kudu_127.17.130.60.keytab
--principal=kudu/127.17.130.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.17.130.60:34935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.17.130.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 00:15:45 UTC on ccycloud.quasar-ehglva.root.comops.site
build id 4581748
I20260812 01:13:45.751948 23090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 01:13:45.752136 23090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 01:13:45.757583 23102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:45.757514 23098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 01:13:45.757524 23097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 01:13:45.757715 23090 server_base.cc:1034] running on GCE node
I20260812 01:13:45.758051 23090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 01:13:45.758386 23090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 01:13:45.759588 23090 hybrid_clock.cc:648] HybridClock initialized: now 1786497225759561 us; error 43 us; skew 500 ppm
I20260812 01:13:45.760206 23076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } attempt: 2
I20260812 01:13:45.760738 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.760568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46874)
0812 01:13:45.760647 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.760693 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 01:13:45.761405 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.761278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46890)
0812 01:13:45.761337 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.761372 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 01:13:45.761545 23076 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.17.130.60:34935: Network error: Client connection negotiation failed: client connection to 127.17.130.60:34935: connect: Connection refused (error 111)
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 01:13:45.762890 23090 init.cc:377] Logged in from keytab as kudu/127.17.130.60@KRBTEST.COM (short username kudu)
I20260812 01:13:45.763403 23090 webserver.cc:460] Webserver started at http://127.17.130.60:33595/ using document root <none> and password file <none>
I20260812 01:13:45.763628 23090 fs_manager.cc:359] Metadata directory not provided
I20260812 01:13:45.763670 23090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 01:13:45.763764 23090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 01:13:45.764809 23090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data/instance:
uuid: "d0e234b54466405fb6c66fbec1341c1e"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "43589f6a3d05c83f0b799071eb86f646"
server_key_iv: "819d85c02f184ca068444c5702ce8683"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.765234 23090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal/instance:
uuid: "d0e234b54466405fb6c66fbec1341c1e"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "43589f6a3d05c83f0b799071eb86f646"
server_key_iv: "819d85c02f184ca068444c5702ce8683"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.766760 23090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:45.767578 23111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 01:13:45.767824 23090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 01:13:45.767940 23090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data,/tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal
uuid: "d0e234b54466405fb6c66fbec1341c1e"
format_stamp: "Formatted at 2026-08-12 01:13:45 on dist-test-slave-tfpf"
server_key: "43589f6a3d05c83f0b799071eb86f646"
server_key_iv: "819d85c02f184ca068444c5702ce8683"
server_key_version: "encryptionkey@0"
I20260812 01:13:45.768024 23090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 01:13:45.775647 23090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 01:13:45.778119 23090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 01:13:45.778242 23090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 01:13:45.782558 23090 rpc_server.cc:307] RPC server started. Bound to: 127.17.130.60:34935
I20260812 01:13:45.782701 23190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.17.130.60:34935 every 8 connection(s)
I20260812 01:13:45.783010 23090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/data/info.pb
I20260812 01:13:45.783340 22960 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } attempt: 2
I20260812 01:13:45.784569 23193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 01:13:45.786554 23193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.62@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:13:45.791261 17928 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskolrujf/build/release/bin/kudu as pid 23090
I20260812 01:13:45.791392 17928 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskolrujf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786496918848844-17928-0/minicluster-data/master-2/wal/instance
I20260812 01:13:45.791618 17928 external_mini_cluster.cc:1428] Setting key 6972b540172fe2152153ba5bc1acdc6c
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:45.793958 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.783736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46902)
0812 01:13:45.783809 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.783823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.783890 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.784948 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.784949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.785175 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.785240 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.785243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.786360 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.786362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.787148 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.787151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.787833 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.787833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.787894 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.788356 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.788362 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.789744 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.792373 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.792374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.792375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.792570 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.792885 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.792886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.792887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.792962 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.793580 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.793581 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.793632 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.793700 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.793856 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 01:13:45.794415 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.783883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46902 (local address 127.17.130.60:34935)
0812 01:13:45.784183 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.784192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.784218 (+ 26us) server_negotiation.cc:407] Connection header received
0812 01:13:45.784721 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.784730 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.784790 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.784914 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.785351 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.786254 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.787244 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.787763 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.787846 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.789890 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.789896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.789896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.789917 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.792054 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.792683 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.792687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.792688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.792783 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.793035 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.793036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.793036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.793416 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.793533 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.793913 (+ 380us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.794182 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":105,"threads_started":1}
I20260812 01:13:45.795797 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.787292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.62:44043 (local address 127.0.0.1:37512)
0812 01:13:45.787466 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.787501 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.787587 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.787754 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.787757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.788176 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.788277 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.788280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.788945 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.788946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.789700 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.789703 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.790099 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.790101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.790205 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.790701 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.790707 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.792408 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.794428 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.794435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.794437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.794719 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.794934 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.794935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.794935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.794994 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.795380 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.795388 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.795476 (+ 88us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.795559 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.795689 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260812 01:13:45.796217 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.787323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.17.130.62:44043)
0812 01:13:45.787368 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.787369 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.787567 (+ 198us) server_negotiation.cc:407] Connection header received
0812 01:13:45.787653 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.787654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.787666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.787732 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.788395 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.788843 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.789804 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.790007 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.790095 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.792503 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.792505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.792505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.792514 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.794337 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.794803 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.794804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.794804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.794863 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.795069 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.795069 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.795070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.795288 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.795349 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.795992 (+ 643us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.796097 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 01:13:45.796394 22960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0: Bootstrap starting.
I20260812 01:13:45.797570 22960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:45.797950 22960 log.cc:824] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0: Log is configured to *not* fsync() on all Append() calls
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:13:45.798954 23193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:45.799165 22960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0: No bootstrap required, opened a new log
I20260812 01:13:45.801220 22960 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.801368 22960 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29ba3c14d1d480a9ad0a42ab0a0cfd0, State: Initialized, Role: FOLLOWER
I20260812 01:13:45.801465 22960 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.801774 23204 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.801857 22960 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:45.801889 23204 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:45.803011 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.792220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.17.130.60:34935)
0812 01:13:45.792386 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.792388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.792707 (+ 319us) server_negotiation.cc:407] Connection header received
0812 01:13:45.792730 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.792731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.792748 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.792821 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.793685 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.794459 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.795551 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.795834 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.795905 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.798852 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.798853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.798854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.798862 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.801382 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.801812 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.801813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.801814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.801875 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.802056 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.802056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.802057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.802179 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.802319 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.802775 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.802861 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
W20260812 01:13:45.805883 23225 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:45.805939 23225 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:45.807835 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.799422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.17.130.61:36873)
0812 01:13:45.799469 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.799471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.799479 (+ 8us) server_negotiation.cc:407] Connection header received
0812 01:13:45.799503 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.799504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.799514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.799569 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.799980 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.800705 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.802060 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.802388 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.802462 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.804775 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.804776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.804777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.804787 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.806474 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.806961 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.806962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.806963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.807028 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.807235 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.807236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.807237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.807391 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.807490 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.807661 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.807741 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 01:13:45.807837 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.799233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.61:36873 (local address 127.0.0.1:59714)
0812 01:13:45.799285 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.799297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.799336 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.799603 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.799605 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.799792 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.799875 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.799878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.800800 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.800802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.801633 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.801635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.802463 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.802464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.802557 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.802992 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.802997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.804675 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.806556 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.806557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.806558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.806824 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.807102 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.807103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.807103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.807175 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.807483 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.807484 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.807534 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.807633 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.807741 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 01:13:45.808245 23193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } has no permanent_uuid. Determining permanent_uuid...
I20260812 01:13:45.805233 22860 master_runner.cc:186] Error getting master registration for 127.17.130.61:36873: OK, instance_id { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" instance_seqno: 1786497225608151 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:13:45.808351 22860 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.62@KRBTEST.COM
I20260812 01:13:45.812788 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.804343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.17.130.62:44043)
0812 01:13:45.804380 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.804382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.804389 (+ 7us) server_negotiation.cc:407] Connection header received
0812 01:13:45.804411 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.804412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.804422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.804481 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.805037 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.805794 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.806846 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.807066 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.807152 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.810056 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.810058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.810058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.810065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.811397 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.811802 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.811803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.811803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.811872 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.812104 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.812105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.812105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.812252 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.812369 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.812603 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.812691 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.60@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:13:45.817227 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.808500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46908)
0812 01:13:45.808791 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.808802 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.808851 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.808976 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.808977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.809157 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.809196 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.809198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.810068 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.810068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.810844 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.810847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.811330 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.811331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.811399 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.811956 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.811960 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.813301 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.815563 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.815564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.815564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.815810 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.816046 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.816046 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.816047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.816124 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.816760 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.816762 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.816814 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.816908 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.817040 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271}
I20260812 01:13:45.817301 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.808579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46908 (local address 127.17.130.60:34935)
0812 01:13:45.808615 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.808617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.808863 (+ 246us) server_negotiation.cc:407] Connection header received
0812 01:13:45.808894 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.808895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.808907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.808969 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.809278 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.809972 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.810937 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.811233 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.811306 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.813389 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.813390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.813390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.813396 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.815457 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.815893 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.815894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.815895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.815957 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.816500 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.816501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.816501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.816660 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.816748 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.816949 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.817138 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":19}
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.17.130.61@KRBTEST.COM
I20260812 01:13:45.818813 23193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: Bootstrap starting.
I20260812 01:13:45.820407 23193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:45.820778 23193 log.cc:824] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:45.821841 23193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: No bootstrap required, opened a new log
I20260812 01:13:45.823045 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.813468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.17.130.61:36873)
0812 01:13:45.813506 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.813508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.813517 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:45.813572 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.813573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.813583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.813647 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.814233 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.814835 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.815515 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.815832 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.815895 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.819038 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.819040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.819041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.819050 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.821228 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.821699 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.821700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.821700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.821778 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.822026 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.822027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.822027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.822164 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.822827 (+ 663us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.822872 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.822957 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:13:45.824445 23193 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.824652 23193 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0e234b54466405fb6c66fbec1341c1e, State: Initialized, Role: FOLLOWER
I20260812 01:13:45.824698 17928 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 01:13:45.824764 23193 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.825124 23228 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.825218 23193 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:45.825268 23228 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:45.825750 22922 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37518
I20260812 01:13:45.825906 22922 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 01:13:45.825965 22922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:45.826917 22922 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.827183 22922 leader_election.cc:288] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [CANDIDATE]: Term 1 election: Requested vote from peers 688a85747fbe4e3793ac67d285f371ff (127.17.130.61:36873), d0e234b54466405fb6c66fbec1341c1e (127.17.130.60:34935)
W20260812 01:13:45.827544 23247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 01:13:45.827651 23247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 01:13:45.827730 23148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d0e234b54466405fb6c66fbec1341c1e"
I20260812 01:13:45.827841 23148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:45.828716 23148 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a29ba3c14d1d480a9ad0a42ab0a0cfd0 in term 1.
I20260812 01:13:45.828966 22891 leader_election.cc:302] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a29ba3c14d1d480a9ad0a42ab0a0cfd0, d0e234b54466405fb6c66fbec1341c1e; no voters:
I20260812 01:13:45.829167 23204 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 01:13:45.829339 23204 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [term 1 LEADER]: Becoming Leader. State: Replica: a29ba3c14d1d480a9ad0a42ab0a0cfd0, State: Running, Role: LEADER
I20260812 01:13:45.829454 23204 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.829921 23250 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29ba3c14d1d480a9ad0a42ab0a0cfd0. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.830035 23250 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:45.830232 23258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 01:13:45.830579 23258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 01:13:45.831219 23090 master_runner.cc:186] Error getting master registration for 127.17.130.61:36873: OK, instance_id { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" instance_seqno: 1786497225608151 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 01:13:45.832823 23090 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 01:13:45.844527 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.830390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.17.130.62:44043)
0812 01:13:45.830432 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.830434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.833318 (+ 2884us) server_negotiation.cc:407] Connection header received
0812 01:13:45.833385 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.833386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.833411 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.833472 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.834226 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.834931 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.835972 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.836273 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.836347 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.840315 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.840317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.840317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.840324 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.842317 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.842801 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.842802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.842803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.842862 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.843221 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.843222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.843222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.843378 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.843457 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.844317 (+ 860us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.844415 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 01:13:45.848176 23085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.831263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.17.130.61:36873)
0812 01:13:45.831304 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.831306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.833474 (+ 2168us) server_negotiation.cc:407] Connection header received
0812 01:13:45.834321 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.834322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.834332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.834361 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.836009 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.836740 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.837786 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.838100 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.838179 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.843352 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.843354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.843355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.843363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.845222 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.845822 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.845824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.845824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.845900 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.846974 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.846975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.846976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.847397 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.847491 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.847964 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.848058 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 01:13:45.850759 23148 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [term 1 FOLLOWER]: Refusing update from remote peer a29ba3c14d1d480a9ad0a42ab0a0cfd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:45.851298 23204 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:45.851470 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.831598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46924 (local address 127.17.130.60:34935)
0812 01:13:45.831634 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.831636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.837893 (+ 6257us) server_negotiation.cc:407] Connection header received
0812 01:13:45.837992 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.837993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.838003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.838033 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.838760 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.840015 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.842087 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.842389 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.842430 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.846817 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.846819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.846820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.846829 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.848725 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.849364 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.849365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.849365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.849433 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.850101 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.850102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.850102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.850259 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.850355 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.851233 (+ 878us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.851373 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:13:45.852679 23076 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } attempt: 3
I20260812 01:13:45.852945 23228 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29ba3c14d1d480a9ad0a42ab0a0cfd0. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.853034 23228 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: This master's current role is: FOLLOWER
Aug 12 01:13:45 dist-test-slave-tfpf krb5kdc[22844](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786497225, etypes {rep=17 tkt=17 ses=17}, kudu/127.17.130.61@KRBTEST.COM for kudu/127.17.130.60@KRBTEST.COM
I20260812 01:13:45.863934 23204 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.864064 23204 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:45.864292 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.852947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.17.130.60:34935 (local address 127.0.0.1:46930)
0812 01:13:45.853029 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 01:13:45.853040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 01:13:45.853149 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 01:13:45.853828 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 01:13:45.853829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 01:13:45.854002 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 01:13:45.854053 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.854054 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.856168 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.856169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.857007 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 01:13:45.857009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.857781 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.857783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 01:13:45.857852 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.858357 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 01:13:45.858362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 01:13:45.860050 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 01:13:45.863187 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.863188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.863188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.863418 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.863632 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 01:13:45.863633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 01:13:45.863633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 01:13:45.863692 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 01:13:45.863972 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 01:13:45.863973 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 01:13:45.864020 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 01:13:45.864066 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 01:13:45.864186 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 01:13:45.864715 23228 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.864800 23228 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:45.865126 23266 mvcc.cc:205] Tried to move back new op lower bound from 7317492637080121344 to 7317492636998180864. Current Snapshot: MvccSnapshot[applied={T|T < 7317492637080121344}]
I20260812 01:13:45.865300 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:45.853252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46930 (local address 127.17.130.60:34935)
0812 01:13:45.853320 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.853322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.853331 (+ 9us) server_negotiation.cc:407] Connection header received
0812 01:13:45.853381 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.853381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.853392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.853485 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.854234 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.855978 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.857122 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:45.857651 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:45.857782 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:45.860156 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 01:13:45.860158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 01:13:45.860158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 01:13:45.860166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 01:13:45.863086 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.863503 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.863504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.863505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.863562 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 01:13:45.863764 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 01:13:45.863765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 01:13:45.863765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 01:13:45.863898 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 01:13:45.864640 (+ 742us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:45.864689 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 01:13:45.864773 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 01:13:45.865284 23261 mvcc.cc:205] Tried to move back new op lower bound from 7317492637080121344 to 7317492636998180864. Current Snapshot: MvccSnapshot[applied={T|T < 7317492637080121344}]
I20260812 01:13:45.867087 23076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: Bootstrap starting.
I20260812 01:13:45.867847 23076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: Neither blocks nor log segments found. Creating new log.
I20260812 01:13:45.868199 23076 log.cc:824] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: Log is configured to *not* fsync() on all Append() calls
I20260812 01:13:45.869247 23076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: No bootstrap required, opened a new log
I20260812 01:13:45.869656 23258 catalog_manager.cc:1347] Generated new cluster ID: f93d8b49e7f34228a35af32f00ee4968
I20260812 01:13:45.869707 23258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 01:13:45.872020 23076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.872414 23076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688a85747fbe4e3793ac67d285f371ff, State: Initialized, Role: FOLLOWER
I20260812 01:13:45.872635 23076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } }
I20260812 01:13:45.873405 23035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 01:13:45.874457 23035 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [term 1 FOLLOWER]: Refusing update from remote peer a29ba3c14d1d480a9ad0a42ab0a0cfd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 01:13:45.874863 23269 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 01:13:45.873430 23036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "688a85747fbe4e3793ac67d285f371ff"
I20260812 01:13:45.875672 23036 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a29ba3c14d1d480a9ad0a42ab0a0cfd0 in term 1.
I20260812 01:13:45.873306 23076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 01:13:45.881299 23274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.881404 23274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:45.881469 23274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29ba3c14d1d480a9ad0a42ab0a0cfd0. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.881518 23274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:45.881556 23274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.881598 23274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff [sys.catalog]: This master's current role is: FOLLOWER
I20260812 01:13:45.882041 23277 mvcc.cc:205] Tried to move back new op lower bound from 7317492637080121344 to 7317492636998180864. Current Snapshot: MvccSnapshot[applied={T|T < 7317492637080121344}]
I20260812 01:13:45.885051 23250 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a29ba3c14d1d480a9ad0a42ab0a0cfd0" member_type: VOTER last_known_addr { host: "127.17.130.62" port: 44043 } } peers { permanent_uuid: "688a85747fbe4e3793ac67d285f371ff" member_type: VOTER last_known_addr { host: "127.17.130.61" port: 36873 } } peers { permanent_uuid: "d0e234b54466405fb6c66fbec1341c1e" member_type: VOTER last_known_addr { host: "127.17.130.60" port: 34935 } } }
I20260812 01:13:45.885516 23250 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0 [sys.catalog]: This master's current role is: LEADER
I20260812 01:13:45.889187 23258 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 01:13:45.889978 23258 catalog_manager.cc:1504] Loading token signing keys...
I20260812 01:13:45.890127 23299 catalog_manager.cc:1259] Loaded cluster ID: f93d8b49e7f34228a35af32f00ee4968
I20260812 01:13:45.890213 23299 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: loading cluster ID for follower catalog manager: success
I20260812 01:13:45.891469 23299 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: acquiring CA information for follower catalog manager: success
W20260812 01:13:45.891680 23299 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260812 01:13:45.912643 23258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29ba3c14d1d480a9ad0a42ab0a0cfd0: Generated new TSK 0
I20260812 01:13:45.912912 23258 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 01:13:45.944715 22968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.940577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37530 (local address 127.17.130.62:44043)
0812 01:13:45.940630 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.940633 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.942279 (+ 1646us) server_negotiation.cc:407] Connection header received
0812 01:13:45.943161 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.943163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.943182 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.943233 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.944574 (+ 1341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37530: BlockingRecv error: recv got EOF from 127.0.0.1:37530 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 01:13:45.945818 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.941208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.17.130.61:36873)
0812 01:13:45.941265 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.941267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.942387 (+ 1120us) server_negotiation.cc:407] Connection header received
0812 01:13:45.945096 (+ 2709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.945099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.945123 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.945165 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.945743 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59732: BlockingRecv error: recv got EOF from 127.0.0.1:59732 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 01:13:45.946854 23200 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.941603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46932 (local address 127.17.130.60:34935)
0812 01:13:45.941649 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.941651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.945193 (+ 3542us) server_negotiation.cc:407] Connection header received
0812 01:13:45.946169 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.946170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.946184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.946217 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.946765 (+ 548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46932: BlockingRecv error: recv got EOF from 127.0.0.1:46932 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:13:45.949363 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.948391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.17.130.61:36873)
0812 01:13:45.948439 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.948441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.948716 (+ 275us) server_negotiation.cc:407] Connection header received
0812 01:13:45.948836 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.948837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.948861 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.948893 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.949298 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59734: BlockingRecv error: recv got EOF from 127.0.0.1:59734 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260812 01:13:45.952724 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.951768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.17.130.61:36873)
0812 01:13:45.951807 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.951809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.952135 (+ 326us) server_negotiation.cc:407] Connection header received
0812 01:13:45.952238 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.952239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.952251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.952283 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.952668 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59740: BlockingRecv error: recv got EOF from 127.0.0.1:59740 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 01:13:45.957157 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.956168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.17.130.61:36873)
0812 01:13:45.956210 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.956212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.956494 (+ 282us) server_negotiation.cc:407] Connection header received
0812 01:13:45.956620 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.956621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.956634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.956665 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.957095 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59742: BlockingRecv error: recv got EOF from 127.0.0.1:59742 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 01:13:45.964076 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.962742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.17.130.61:36873)
0812 01:13:45.962811 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.962813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.963386 (+ 573us) server_negotiation.cc:407] Connection header received
0812 01:13:45.963508 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.963509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.963524 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.963555 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.964012 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59758: BlockingRecv error: recv got EOF from 127.0.0.1:59758 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 01:13:45.974601 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.973607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.17.130.61:36873)
0812 01:13:45.973655 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.973657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.973964 (+ 307us) server_negotiation.cc:407] Connection header received
0812 01:13:45.974077 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.974078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.974091 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.974123 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.974538 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59762: BlockingRecv error: recv got EOF from 127.0.0.1:59762 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 01:13:45.993083 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:45.992079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59772 (local address 127.17.130.61:36873)
0812 01:13:45.992129 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:45.992131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:45.992437 (+ 306us) server_negotiation.cc:407] Connection header received
0812 01:13:45.992547 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:45.992548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:45.992561 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:45.992592 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:45.992993 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59772: BlockingRecv error: recv got EOF from 127.0.0.1:59772 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260812 01:13:46.028439 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:46.027376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59778 (local address 127.17.130.61:36873)
0812 01:13:46.027429 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:46.027432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:46.027718 (+ 286us) server_negotiation.cc:407] Connection header received
0812 01:13:46.027820 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:46.027826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:46.027838 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:46.027891 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:46.028374 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59778: BlockingRecv error: recv got EOF from 127.0.0.1:59778 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260812 01:13:46.094878 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:46.094056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.17.130.61:36873)
0812 01:13:46.094129 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:46.094131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:46.094242 (+ 111us) server_negotiation.cc:407] Connection header received
0812 01:13:46.094336 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:46.094338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:46.094357 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:46.094425 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:46.094793 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59794: BlockingRecv error: recv got EOF from 127.0.0.1:59794 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 01:13:46.225594 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:46.224298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.17.130.61:36873)
0812 01:13:46.224367 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:46.224369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:46.224411 (+ 42us) server_negotiation.cc:407] Connection header received
0812 01:13:46.224476 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:46.224476 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:46.224487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:46.224772 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:46.225541 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59802: BlockingRecv error: recv got EOF from 127.0.0.1:59802 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 01:13:46.484329 23085 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 01:13:46.483286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43304 (local address 127.17.130.61:36873)
0812 01:13:46.483395 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:46.483398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:46.483525 (+ 127us) server_negotiation.cc:407] Connection header received
0812 01:13:46.483627 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:46.483628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:46.483647 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:46.483716 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 01:13:46.484250 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43304: BlockingRecv error: recv got EOF from 127.0.0.1:43304 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 01:13:46.828251 23247 catalog_manager.cc:1259] Loaded cluster ID: f93d8b49e7f34228a35af32f00ee4968
I20260812 01:13:46.828348 23247 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: loading cluster ID for follower catalog manager: success
I20260812 01:13:46.829313 23247 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: acquiring CA information for follower catalog manager: success
I20260812 01:13:46.829691 23247 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d0e234b54466405fb6c66fbec1341c1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:13:46.892482 23299 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 688a85747fbe4e3793ac67d285f371ff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 01:13:47.000914 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:46.997779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.17.130.61:36873)
0812 01:13:46.997939 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:46.997942 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:46.998057 (+ 115us) server_negotiation.cc:407] Connection header received
0812 01:13:46.998177 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:46.998178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:46.998249 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:46.998351 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:46.998539 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:46.999093 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:46.999961 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.000194 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.000285 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:47.000496 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:47.000627 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:47.000729 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:47.000773 (+ 44us) server_negotiation.cc:299] Negotiation successful
0812 01:13:47.000799 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":57,"threads_started":1}
I20260812 01:13:47.005700 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:47.003153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.17.130.60:34935)
0812 01:13:47.003279 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:47.003281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:47.003291 (+ 10us) server_negotiation.cc:407] Connection header received
0812 01:13:47.003337 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:47.003338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:47.003394 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:47.003483 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:47.003594 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.004082 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.004825 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.005049 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.005165 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:47.005341 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:47.005451 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:47.005540 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:47.005582 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 01:13:47.005609 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":51,"threads_started":1}
I20260812 01:13:47.010478 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:47.007435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.17.130.60:34935)
0812 01:13:47.007475 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:47.007477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:47.007620 (+ 143us) server_negotiation.cc:407] Connection header received
0812 01:13:47.007676 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:47.007678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:47.007688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:47.007740 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:47.007905 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.008604 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.009663 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.009944 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.010002 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:47.010215 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:47.010243 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:47.010270 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:47.010370 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 01:13:47.010397 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 01:13:47.010656 23419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:47.007381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.17.130.62:44043)
0812 01:13:47.007534 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:47.007536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:47.007767 (+ 231us) server_negotiation.cc:407] Connection header received
0812 01:13:47.007840 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:47.007841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:47.007895 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:47.007981 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:47.008100 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.008702 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.009588 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.009894 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.009996 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:47.010196 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:47.010309 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:47.010518 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:47.010538 (+ 20us) server_negotiation.cc:299] Negotiation successful
0812 01:13:47.010562 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260812 01:13:47.010947 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 01:13:47.007695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43324 (local address 127.17.130.61:36873)
0812 01:13:47.007741 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 01:13:47.007744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 01:13:47.007906 (+ 162us) server_negotiation.cc:407] Connection header received
0812 01:13:47.008031 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 01:13:47.008031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 01:13:47.008042 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 01:13:47.008099 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 01:13:47.008299 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.009000 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.009925 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 01:13:47.010233 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 01:13:47.010304 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 01:13:47.010454 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 01:13:47.010486 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 01:13:47.010526 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 01:13:47.010839 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 01:13:47.010871 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 01:13:47.012825 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22860
I20260812 01:13:47.030055 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 22976
I20260812 01:13:47.043352 17928 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskolrujf/build/release/bin/kudu with pid 23090
2026-08-12T01:13:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3190 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6559 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (308199 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST